-- Logs begin at Mon 2022-10-17 06:24:17 UTC, end at Mon 2022-10-17 07:24:09 UTC. -- Oct 17 06:49:22.837608 dv-cinder-845069-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 17 06:49:22.845610 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 17 06:49:22 2022] *** Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: nodename: dv-cinder-845069-5-iscsi Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: machine: x86_64 Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: clock source: unix Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: pcre jit disabled Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: detected number of CPU cores: 4 Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: current working directory: / Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: detected binary path: /usr/bin/uwsgi-core Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: your processes number limit is 47680 Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: your memory page size is 4096 bytes Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: detected max file descriptor number: 2048 Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: lock engine: pthread robust mutexes Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: thunder lock: enabled Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 17 06:49:22.850012 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 17 06:49:22.887516 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: Python main interpreter initialized at 0x556332d6b9c0 Oct 17 06:49:22.887516 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: python threads support enabled Oct 17 06:49:22.887516 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: your server socket listen backlog is limited to 100 connections Oct 17 06:49:22.887516 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: your mercy for graceful operations on workers is 90 seconds Oct 17 06:49:22.887887 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: mapped 403077 bytes (393 KB) for 2 cores Oct 17 06:49:22.887979 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: *** Operational MODE: preforking *** Oct 17 06:49:22.888217 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: *** uWSGI is running in multiple interpreter mode *** Oct 17 06:49:22.888217 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: spawned uWSGI master process (pid: 81471) Oct 17 06:49:22.888420 dv-cinder-845069-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 17 06:49:22.889835 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 1 (pid: 81487, cores: 1) Oct 17 06:49:22.889835 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 2 (pid: 81488, cores: 1) Oct 17 06:49:22.889977 dv-cinder-845069-5-iscsi devstack@keystone.service[81471]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 17 06:49:27.104690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 06:49:27.104690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: from cryptography.utils import int_from_bytes Oct 17 06:49:27.105446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 06:49:27.105446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: from cryptography.utils import int_from_bytes Oct 17 06:49:27.105574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 06:49:27.105574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: from cryptography.utils import int_from_bytes Oct 17 06:49:27.105574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 06:49:27.105574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: from cryptography.utils import int_from_bytes Oct 17 06:49:27.154903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 06:49:27.155096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 06:49:27.155250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 06:49:27.155355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] command line args: [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 06:49:27.155437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 06:49:27.155561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 06:49:27.155649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] command line args: [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 06:49:27.155727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ================================================================================ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 06:49:27.155872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 06:49:27.155971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] admin_token = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ================================================================================ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 06:49:27.156226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config_dir = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] admin_token = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config_dir = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config_source = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] debug = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config_source = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.156965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] debug = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.157961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_config_append = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_config_append = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_dir = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_dir = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.158999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.159957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.160962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.161974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.162130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.162206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.162401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.162629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.162806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] publish_errors = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.162881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] publish_errors = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.163984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] standard_threads = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] standard_threads = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_journal = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.164983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_json = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_journal = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_stderr = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_json = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_syslog = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_stderr = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165598 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_syslog = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 06:49:27.165897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.166858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.167977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168521 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.168983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.169146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.169223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.169385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.169460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.169616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.170204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.170377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.170626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.170867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.171888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172818 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.172971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.173979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.174903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.175985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176658 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.176953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.177865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.178925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.179961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.180886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.181913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.182920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.183933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184123 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.184939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.185015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.185169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.185253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.185955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.186963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.187921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.188067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.188145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.188302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.188381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.188536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.188688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.188762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.189996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.190906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.191870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.external = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192744 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.192974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.password = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.external = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.token = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193744 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.193987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.password = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194213 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.token = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.194901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.195863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196482 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196722 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.196962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.197921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.198958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.199930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.200864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.201845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.202925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.203973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.204878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205348 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.205979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206214 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.206986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.207934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208312 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.208873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209012 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.209947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.210944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.211946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.212969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.213890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.214975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.215923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.216945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.217878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.218959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.219999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.220927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.221858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222009 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.222943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223023 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.223993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.224994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.225997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.226944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227916 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.227993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.228986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.229112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.229227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.229357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.229469 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.229710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.229787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.229952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.230984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.231933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.232086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.232703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.232841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.232921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.233994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.234930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.235981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.236912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.237850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.238965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.239919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240230 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.240916 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241157 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.241845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.242001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.242088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.242228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.242307 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.242379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 06:49:27.242470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.242697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.242933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.243164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 06:49:27.243353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 06:49:27.272197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.272420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.272641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.273359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.273593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.273815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.281501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.282669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.310569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.310795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.311155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.311416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.365263 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.365400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.414107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.414330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.414659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.414890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.569843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.570925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.571008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.571135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.571377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.594389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.594606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.594698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.594828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.594927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.595914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.606841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.606992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.620969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.621099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.621183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.621311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.621394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.621519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 06:49:27.637879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.638088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 06:49:27.638383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.638466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.638594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 06:49:27.638676 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 06:49:27.638905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.638988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.639116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 06:49:27.639228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.639425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.639527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.639682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.639763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.639935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.640189 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.649198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.649479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.649625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.649751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.649900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.650034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.650167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.650446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.657524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.657858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.660427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.660719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.660804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.661001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.661084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.661278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.661358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.661561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.661642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.661824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.661905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662176 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.662967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.663149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.663245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.663437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.686146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.686296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.687629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.687776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.687914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.692805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.692958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.693090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.693239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.693375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.693519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.699656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.699775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.699941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.700058 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.705531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.705630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.705828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.705912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.706115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.706199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.706404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.706555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.706682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.706766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.706959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.707051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.707252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.707338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.707527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.707610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.723346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.723449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.723626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.723709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.727421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.727506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.730581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.730665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.730878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.730961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.735104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.735240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.738158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.738313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.741336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.741523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.741618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.741822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.741905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.742105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.742187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.742387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.742471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.742664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.742747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.742950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.758719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.759005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.759088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.759301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.759387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.759569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.759664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.759851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.766419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.766677 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.767997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.768232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.769525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.769726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.769808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.769998 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.772441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.772627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.772737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.772908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.773019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.773185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.773300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.773474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.773698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.773860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.785305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.785412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.785589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.785674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.785869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.785961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.786151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.786235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.786524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.786606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.786802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.786893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.787074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.787156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.802430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.802588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.805317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.805457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.808245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.808395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.809700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.809840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.811689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.811826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.811953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.812079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.812351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.812533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.818850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.818989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.820320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.820452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.823160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.823308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.823577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.823706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.823848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.823983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.824125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.824251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.832371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.832511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.832664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.832802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.832929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.833055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.839732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.839905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 06:49:27.840044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.840188 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.840319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.840448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.840574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.840699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.840832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.840959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.841084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.841236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.841363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.841545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.841696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.841831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.841959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.842093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.842221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.842348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.842580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.842708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.842876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.843004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.843165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.843317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.843443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.843574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 06:49:27.877414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556332d6b9c0 pid: 81488 (default app) Oct 17 06:49:27.877515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556332d6b9c0 pid: 81487 (default app) Oct 17 06:49:27.880597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec13f-2231-4e6c-9cd7-be0070e873da None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:27.880878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec13f-2231-4e6c-9cd7-be0070e873da None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:27.881149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec13f-2231-4e6c-9cd7-be0070e873da None None] PATH_INFO: `/v3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:27.885266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Mon Oct 17 06:49:27 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:34.643438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3160c5-72c4-4942-a5b4-938526112233 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:34.643918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3160c5-72c4-4942-a5b4-938526112233 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:34.643918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3160c5-72c4-4942-a5b4-938526112233 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:34.648166 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:49:34.653260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:34.654363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:34.654363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:34.723997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: INFO dbcounter [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] Registered counter for database keystone Oct 17 06:49:34.742275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG oslo_db.sqlalchemy.engines [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81488) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 06:49:34.748330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writer thread running {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 06:49:34.924256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG passlib.handlers.bcrypt [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81488) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 06:49:34.924654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG passlib.handlers.bcrypt [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81488) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 06:49:34.983580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:34.985157 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:35.108546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:49:35.392761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3160c5-72c4-4942-a5b4-938526112233 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:35.460973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: INFO dbcounter [None req-ad3160c5-72c4-4942-a5b4-938526112233 None None] Registered counter for database keystone Oct 17 06:49:35.482259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG oslo_db.sqlalchemy.engines [None req-ad3160c5-72c4-4942-a5b4-938526112233 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81487) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 06:49:35.488542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writer thread running {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 06:49:35.604454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3160c5-72c4-4942-a5b4-938526112233 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:35.609132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ded60-a7c7-4561-934f-6802f8417f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:35.609745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ded60-a7c7-4561-934f-6802f8417f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:35.610019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ded60-a7c7-4561-934f-6802f8417f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:35.610286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ded60-a7c7-4561-934f-6802f8417f14 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:35.618277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ded60-a7c7-4561-934f-6802f8417f14 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:35.668962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8d3ded60-a7c7-4561-934f-6802f8417f14 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 06:49:35.669983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ded60-a7c7-4561-934f-6802f8417f14 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:35.678871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Mon Oct 17 06:49:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 06:49:35.685452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:35.702208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03a3ed-1d50-4981-9d4a-5dcc6ea0c5f5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:35.706475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396239c7-5260-4ec1-bdb2-bf63bbdfe077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:35.707588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396239c7-5260-4ec1-bdb2-bf63bbdfe077 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:35.707868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396239c7-5260-4ec1-bdb2-bf63bbdfe077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:35.708136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396239c7-5260-4ec1-bdb2-bf63bbdfe077 admin admin] PATH_INFO: `/v3/projects/86c1b532428347929d42c84633169ba2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:35.709688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396239c7-5260-4ec1-bdb2-bf63bbdfe077 admin admin] RBAC: Authorizing `identity:get_project(project_id=86c1b532428347929d42c84633169ba2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:35.762063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING keystone.common.rbac_enforcer.enforcer [None req-396239c7-5260-4ec1-bdb2-bf63bbdfe077 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 06:49:35.762406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396239c7-5260-4ec1-bdb2-bf63bbdfe077 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:35.766851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Mon Oct 17 06:49:35 2022] GET /identity/v3/projects/86c1b532428347929d42c84633169ba2 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:37.839548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47ee81-f1f9-4872-98b3-63bf3f83ae47 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:37.840057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47ee81-f1f9-4872-98b3-63bf3f83ae47 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:37.840057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47ee81-f1f9-4872-98b3-63bf3f83ae47 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:37.844170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:49:37.849902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2dfc6-c98d-451c-8fed-125b7e7c1273 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:37.850261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2dfc6-c98d-451c-8fed-125b7e7c1273 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:37.850434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2dfc6-c98d-451c-8fed-125b7e7c1273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:37.869887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c9f2dfc6-c98d-451c-8fed-125b7e7c1273 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:37.871405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c9f2dfc6-c98d-451c-8fed-125b7e7c1273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:37.879082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:49:38.143658 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47ee81-f1f9-4872-98b3-63bf3f83ae47 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:38.155963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47ee81-f1f9-4872-98b3-63bf3f83ae47 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:38.160083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9d1c3-986b-431b-bcb4-7092fc300d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:38.160677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9d1c3-986b-431b-bcb4-7092fc300d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:38.160953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9d1c3-986b-431b-bcb4-7092fc300d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:38.161220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9d1c3-986b-431b-bcb4-7092fc300d77 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:38.168686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9d1c3-986b-431b-bcb4-7092fc300d77 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:38.171026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9d1c3-986b-431b-bcb4-7092fc300d77 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:38.179606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Mon Oct 17 06:49:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 06:49:38.185253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2dfc6-c98d-451c-8fed-125b7e7c1273 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:38.194464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2dfc6-c98d-451c-8fed-125b7e7c1273 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:38.198764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e1aa0-676f-4174-b056-63709bb87f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:38.199408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e1aa0-676f-4174-b056-63709bb87f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:38.199691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e1aa0-676f-4174-b056-63709bb87f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:38.199966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e1aa0-676f-4174-b056-63709bb87f2f admin admin] PATH_INFO: `/v3/users/be01c4fc32bc463e9e081f13d9758f92` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:38.201312 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e1aa0-676f-4174-b056-63709bb87f2f admin admin] RBAC: Authorizing `identity:get_user(user_id=be01c4fc32bc463e9e081f13d9758f92)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:38.203131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e1aa0-676f-4174-b056-63709bb87f2f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:38.204660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Mon Oct 17 06:49:38 2022] GET /identity/v3/users/be01c4fc32bc463e9e081f13d9758f92 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:49:40.359153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa01b-d126-470a-a6f0-416b69e20a30 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:40.359739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa01b-d126-470a-a6f0-416b69e20a30 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:40.359739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa01b-d126-470a-a6f0-416b69e20a30 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:40.361035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:40.366345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440baf2-9a55-4951-96cf-67251eb1e0ec None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:40.366759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440baf2-9a55-4951-96cf-67251eb1e0ec None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:40.366886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440baf2-9a55-4951-96cf-67251eb1e0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:40.384842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1440baf2-9a55-4951-96cf-67251eb1e0ec None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:40.386016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1440baf2-9a55-4951-96cf-67251eb1e0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:40.435863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6871d99-3091-4634-bee0-547200a5b687 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:40.436068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6871d99-3091-4634-bee0-547200a5b687 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:40.436338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6871d99-3091-4634-bee0-547200a5b687 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:40.437637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:49:40.443313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:40.443590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:40.443859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:40.466056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:49:40.485196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG passlib.handlers.bcrypt [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81487) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 06:49:40.485588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG passlib.handlers.bcrypt [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81487) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 06:49:40.534783 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:40.536360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:40.632030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:40.731311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440baf2-9a55-4951-96cf-67251eb1e0ec None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:40.756158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:40.834529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440baf2-9a55-4951-96cf-67251eb1e0ec None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:40.867785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:40.868407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:40.868681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:40.868950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:40.879851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:49:40.889015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:40.890069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:49:40.891160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:40.892141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:49:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:49:40.895983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:40.898002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:40.978770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:40.995780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777290f6-7dae-4d2c-b5fe-293e1f1664e4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:40.997232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e7225-c593-4816-994b-b68a5593730a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:40.998930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:41.030555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:41.031149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:41.031449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:41.031716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:41.032378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:41.032480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:41.033299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:41.033581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:41.033763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:41.033852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:41.034675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:41.034975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:41.036423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:41.043136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.sql.core [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:49:41.043136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:49:41.043136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [parameters: {'id': 'd1d1a770875242ce9cc1f9c76f3c2d8b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 06:49:41.043136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81488) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:49:41.048718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:41.054105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:49:41.061371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 06:49:41.062280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application yield resource Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise self.value Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application session.commit() Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:49:41.064339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:49:41.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:49:41.066409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:49:41.067298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [parameters: {'id': 'd1d1a770875242ce9cc1f9c76f3c2d8b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 06:49:41.068264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:41.069102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 8/16] 10.222.0.36 () {60 vars in 1137 bytes} [Mon Oct 17 06:49:40 2022] POST /identity/v3/domains => generated 190 bytes in 165 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:49:41.070421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:41.159656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe67d96-835c-47e3-af06-8d5e5d8f0394 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:41.167652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac9c7c-cfd0-49c5-9607-6bbf00df7722 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:41.196654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d563f-145c-468d-9719-19b547827df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:41.197503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d563f-145c-468d-9719-19b547827df5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:41.197781 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d563f-145c-468d-9719-19b547827df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:41.198055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d563f-145c-468d-9719-19b547827df5 None admin] PATH_INFO: `/v3/users/be01c4fc32bc463e9e081f13d9758f92` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:41.199558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d563f-145c-468d-9719-19b547827df5 None admin] RBAC: Authorizing `identity:get_user(user_id=be01c4fc32bc463e9e081f13d9758f92)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:41.201007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d563f-145c-468d-9719-19b547827df5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:41.202753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:49:41 2022] GET /identity/v3/users/be01c4fc32bc463e9e081f13d9758f92 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:49:41.203766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:41.204385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:41.204660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:41.204930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:41.207915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d563f-145c-468d-9719-19b547827df5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:41.215233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:41.217224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:41.224158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:49:41.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:41.234628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:49:41.235891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:49:41.235891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:49:41.235891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:41.235891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:49:41.240654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:41.242360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:41.308095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d563f-145c-468d-9719-19b547827df5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:41.339361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4122-59f0-4181-99b8-ad3f39c9129d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:41.342033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:41.342667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:41.342948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:41.343239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:41.344350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:41.345797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:41.347366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:49:41.352841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:41.374091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33d11c-057d-4411-848b-82d93b724acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:41.374697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d11c-057d-4411-848b-82d93b724acd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:41.374971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d11c-057d-4411-848b-82d93b724acd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:41.375267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d11c-057d-4411-848b-82d93b724acd None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:41.375907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33d11c-057d-4411-848b-82d93b724acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:41.376208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33d11c-057d-4411-848b-82d93b724acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:41.377593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33d11c-057d-4411-848b-82d93b724acd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:41.395361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:49:41.458705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2aee4-448e-467f-a2e6-cfbf3074df5f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:41.460405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:41.495694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d7c3c-6433-4878-88a2-2ea05e249336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:41.496366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7c3c-6433-4878-88a2-2ea05e249336 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:41.496644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7c3c-6433-4878-88a2-2ea05e249336 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:41.496920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7c3c-6433-4878-88a2-2ea05e249336 None admin] PATH_INFO: `/v3/domains/default/users/be01c4fc32bc463e9e081f13d9758f92/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:41.499513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d7c3c-6433-4878-88a2-2ea05e249336 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=be01c4fc32bc463e9e081f13d9758f92, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:41.501007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d7c3c-6433-4878-88a2-2ea05e249336 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:41.619512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 11/21] 10.222.0.36 () {58 vars in 1361 bytes} [Mon Oct 17 06:49:41 2022] PUT /identity/v3/domains/default/users/be01c4fc32bc463e9e081f13d9758f92/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:49:44.060536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fbaca-6838-43b0-a94e-3468f4f06409 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.061038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fbaca-6838-43b0-a94e-3468f4f06409 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.061110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fbaca-6838-43b0-a94e-3468f4f06409 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.062656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:44.069121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e722a28-4135-40f9-adb9-80c2c1166354 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.069546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e722a28-4135-40f9-adb9-80c2c1166354 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.069714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e722a28-4135-40f9-adb9-80c2c1166354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.070907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0337cf-3ec7-4f62-a3a6-d54b51d94ea3 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.071185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0337cf-3ec7-4f62-a3a6-d54b51d94ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.071483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0337cf-3ec7-4f62-a3a6-d54b51d94ea3 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.072984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 12/23] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:44.077976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de60acc-7076-48e6-b984-7bc35476ce4a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.078257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de60acc-7076-48e6-b984-7bc35476ce4a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.078530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de60acc-7076-48e6-b984-7bc35476ce4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.112483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7e722a28-4135-40f9-adb9-80c2c1166354 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:44.117461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7e722a28-4135-40f9-adb9-80c2c1166354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:44.120492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2de60acc-7076-48e6-b984-7bc35476ce4a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:44.121668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2de60acc-7076-48e6-b984-7bc35476ce4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:44.231590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:44.234940 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 12/24] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:49:44.236527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aa59c-76db-4764-b117-6b9339ef0a5a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.237102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aa59c-76db-4764-b117-6b9339ef0a5a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.237571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aa59c-76db-4764-b117-6b9339ef0a5a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.242407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 13/25] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:44.248324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e82aa-b957-413f-be80-13d6c970abd6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.248860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e82aa-b957-413f-be80-13d6c970abd6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.249332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e82aa-b957-413f-be80-13d6c970abd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.269019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 13/26] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:44.269568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-175e82aa-b957-413f-be80-13d6c970abd6 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:44.270963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-175e82aa-b957-413f-be80-13d6c970abd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:44.380654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 14/27] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:49:44.646719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de60acc-7076-48e6-b984-7bc35476ce4a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:44.664691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e82aa-b957-413f-be80-13d6c970abd6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:44.710903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:44.851396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de60acc-7076-48e6-b984-7bc35476ce4a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:44.860873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e82aa-b957-413f-be80-13d6c970abd6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:44.890552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:44.891306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.891658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.891997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.892735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:44.894210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:44.904235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245faed-f37f-4b44-9454-050449abe27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:44.905206 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245faed-f37f-4b44-9454-050449abe27a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.905654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245faed-f37f-4b44-9454-050449abe27a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.906099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245faed-f37f-4b44-9454-050449abe27a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.926938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245faed-f37f-4b44-9454-050449abe27a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:44.928844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245faed-f37f-4b44-9454-050449abe27a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7245faed-f37f-4b44-9454-050449abe27a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:49:44.947960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:44.948858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:49:44.949675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:49:44.949675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:49:44.949675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:49:44.951487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 15/28] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:49:44.952976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cef18-d4b9-470a-b905-8b6de7351b9c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:44.953445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cef18-d4b9-470a-b905-8b6de7351b9c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:44.953903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cef18-d4b9-470a-b905-8b6de7351b9c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:44.955822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 16/29] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:44.956848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cef18-d4b9-470a-b905-8b6de7351b9c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:45.020920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 14/30] 10.222.0.36 () {60 vars in 1131 bytes} [Mon Oct 17 06:49:44 2022] POST /identity/v3/roles => generated 219 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:49:45.022495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:45.066665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:45.122902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cef18-d4b9-470a-b905-8b6de7351b9c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:45.138195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:45.202697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27c091-1178-42d7-9f13-bbddc004de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:45.203372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c091-1178-42d7-9f13-bbddc004de13 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.203651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c091-1178-42d7-9f13-bbddc004de13 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.203918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c091-1178-42d7-9f13-bbddc004de13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.229706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27c091-1178-42d7-9f13-bbddc004de13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-5b27c091-1178-42d7-9f13-bbddc004de13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:49:45.238719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:45.239828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:49:45.240584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:49:45.241161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828c16-09c8-4e56-8281-c7711a0016e4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:45.243425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 17/31] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:49:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:49:45.244882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1b6d1-574c-4ee2-bdb0-bd81ece40f05 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.245382 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1b6d1-574c-4ee2-bdb0-bd81ece40f05 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.245846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1b6d1-574c-4ee2-bdb0-bd81ece40f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.271451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bbb1b6d1-574c-4ee2-bdb0-bd81ece40f05 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:45.272788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bbb1b6d1-574c-4ee2-bdb0-bd81ece40f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:45.296530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:45.297219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.297497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.297770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.298573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:45.298879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:45.300605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:45.316436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 15/32] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:45.317699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:45.373857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 18/33] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:45.375069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1b6d1-574c-4ee2-bdb0-bd81ece40f05 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:45.453274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6e0da-8b08-4f54-8edb-9c02ede22a7e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:45.464931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:45.466904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:45.498814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188c297-5640-4f3c-9d35-3bc30fb56bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:45.499769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c297-5640-4f3c-9d35-3bc30fb56bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.500062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c297-5640-4f3c-9d35-3bc30fb56bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.500341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c297-5640-4f3c-9d35-3bc30fb56bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.501289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188c297-5640-4f3c-9d35-3bc30fb56bd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:45.501602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188c297-5640-4f3c-9d35-3bc30fb56bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:45.503246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188c297-5640-4f3c-9d35-3bc30fb56bd5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:45.509589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1b6d1-574c-4ee2-bdb0-bd81ece40f05 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:45.518801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 16/34] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:45.520139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577e54e-613d-4d23-8e54-1ea9af9ecf5b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.520494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577e54e-613d-4d23-8e54-1ea9af9ecf5b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.520773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577e54e-613d-4d23-8e54-1ea9af9ecf5b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.522390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 17/35] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:45.527865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677ab6-ae83-4c7e-b818-4e413b428fc8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.528721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677ab6-ae83-4c7e-b818-4e413b428fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.528994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677ab6-ae83-4c7e-b818-4e413b428fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.547320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8a677ab6-ae83-4c7e-b818-4e413b428fc8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:45.549022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a677ab6-ae83-4c7e-b818-4e413b428fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:45.550299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:45.551329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.551923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.552412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.553239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:45.554940 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:45.589285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 19/36] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:49:45 2022] POST /identity/v3/projects => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:49:45.591290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:45.624037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 18/37] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:45.686177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a677ab6-ae83-4c7e-b818-4e413b428fc8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:45.713361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a8e5e-5d7e-442f-ba5c-970f3c72c149 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:45.736618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:45.762256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:45.763442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.764020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.764486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] PATH_INFO: `/v3/users/be01c4fc32bc463e9e081f13d9758f92` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.766085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=be01c4fc32bc463e9e081f13d9758f92)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:45.767941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:45.769819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 20/38] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:49:45 2022] GET /identity/v3/users/be01c4fc32bc463e9e081f13d9758f92 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:45.775344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:45.878625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:45.881695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a677ab6-ae83-4c7e-b818-4e413b428fc8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:45.920262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca712-ea18-4316-8002-4a06cdbdf7f8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:45.927267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f079869-7872-4aea-af41-94342719466a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:45.928139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f079869-7872-4aea-af41-94342719466a None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.928577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f079869-7872-4aea-af41-94342719466a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.928997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f079869-7872-4aea-af41-94342719466a None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.929809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f079869-7872-4aea-af41-94342719466a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:45.931394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f079869-7872-4aea-af41-94342719466a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:45.946155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 19/39] 10.222.0.36 () {60 vars in 1131 bytes} [Mon Oct 17 06:49:45 2022] POST /identity/v3/roles => generated 225 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:49:45.947460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c833-e5e7-4f5f-b2f6-7ee64f4ccc1a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.947741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c833-e5e7-4f5f-b2f6-7ee64f4ccc1a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.948007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c833-e5e7-4f5f-b2f6-7ee64f4ccc1a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.954373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 20/40] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:45.955587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfea81-b378-48be-837a-aa027c8b1025 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.955878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfea81-b378-48be-837a-aa027c8b1025 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.956135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfea81-b378-48be-837a-aa027c8b1025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.966467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:45.967040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:45.967377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:45.967632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:45.968961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:45.970510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:45.972433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 21/41] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:49:45.977755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:46.043114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:46.085054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b2cfea81-b378-48be-837a-aa027c8b1025 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:46.086283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b2cfea81-b378-48be-837a-aa027c8b1025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:46.100851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef09e-2a56-4d7f-9410-3b31a6d1277e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:46.136852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:46.137600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:46.137958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:46.138944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:46.138944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:46.140323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:46.160883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 22/42] 10.222.0.36 () {60 vars in 1131 bytes} [Mon Oct 17 06:49:45 2022] POST /identity/v3/roles => generated 223 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:49:46.161464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:46.167673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 21/43] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:46.289346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2de38-400c-48b2-b716-45fb91c05718 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:46.304259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:46.328935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:46.329551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:46.329835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:46.330114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:46.330849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.domain.id=default, user.id=be01c4fc32bc463e9e081f13d9758f92) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:46.331157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:46.332648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:46.342174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 23/44] 10.222.0.36 () {56 vars in 1322 bytes} [Mon Oct 17 06:49:46 2022] GET /identity/v3/role_assignments?user.id=be01c4fc32bc463e9e081f13d9758f92&scope.domain.id=default&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 530 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:46.445935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfea81-b378-48be-837a-aa027c8b1025 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:46.539473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:46.550292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfea81-b378-48be-837a-aa027c8b1025 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:46.584788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d89eb-38a3-445a-90a8-ff5912b9030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:46.585420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d89eb-38a3-445a-90a8-ff5912b9030a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:46.585701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d89eb-38a3-445a-90a8-ff5912b9030a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:46.585974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d89eb-38a3-445a-90a8-ff5912b9030a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:46.587062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d89eb-38a3-445a-90a8-ff5912b9030a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:46.588565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d89eb-38a3-445a-90a8-ff5912b9030a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:46.590117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 22/45] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:46.595731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:46.690194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6b7d3-ffe2-4aad-95a5-8f7a788e254a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:46.722740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34e689-5bfe-44d9-9a65-3c23c75e0ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:46.723368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34e689-5bfe-44d9-9a65-3c23c75e0ad5 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:46.723649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34e689-5bfe-44d9-9a65-3c23c75e0ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:46.723920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34e689-5bfe-44d9-9a65-3c23c75e0ad5 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:46.724562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34e689-5bfe-44d9-9a65-3c23c75e0ad5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:46.725994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34e689-5bfe-44d9-9a65-3c23c75e0ad5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:46.754046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 24/46] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:49:46 2022] POST /identity/v3/projects => generated 312 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:49:48.888345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c251f9-b585-455b-a6d2-cfa4b36f45cb None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:48.888882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c251f9-b585-455b-a6d2-cfa4b36f45cb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:48.888882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c251f9-b585-455b-a6d2-cfa4b36f45cb None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:48.890236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 23/47] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:48.895459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a90f15-2d99-45e8-bf08-5a5ea2b9319a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:48.895884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a90f15-2d99-45e8-bf08-5a5ea2b9319a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:48.896003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a90f15-2d99-45e8-bf08-5a5ea2b9319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:48.913520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-45a90f15-2d99-45e8-bf08-5a5ea2b9319a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:48.914690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-45a90f15-2d99-45e8-bf08-5a5ea2b9319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:48.937276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:48.979544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 25/48] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:49.248554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c251f9-b585-455b-a6d2-cfa4b36f45cb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:49.346629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c251f9-b585-455b-a6d2-cfa4b36f45cb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:49.379060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc766127-b7ef-40c4-bc31-0f74432512df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:49.379701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766127-b7ef-40c4-bc31-0f74432512df None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:49.379983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766127-b7ef-40c4-bc31-0f74432512df None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:49.380257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766127-b7ef-40c4-bc31-0f74432512df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:49.381363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc766127-b7ef-40c4-bc31-0f74432512df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:49.382803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc766127-b7ef-40c4-bc31-0f74432512df None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:49.384373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 24/49] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:49.390561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a90f15-2d99-45e8-bf08-5a5ea2b9319a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:49.483245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a90f15-2d99-45e8-bf08-5a5ea2b9319a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:49.515794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae49cb0-9b10-4924-9274-fd0fbbe39d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:49.516388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae49cb0-9b10-4924-9274-fd0fbbe39d66 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:49.516689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae49cb0-9b10-4924-9274-fd0fbbe39d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:49.516957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae49cb0-9b10-4924-9274-fd0fbbe39d66 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:49.517590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae49cb0-9b10-4924-9274-fd0fbbe39d66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:49.519035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae49cb0-9b10-4924-9274-fd0fbbe39d66 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:49.543321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 26/50] 10.222.0.36 () {60 vars in 1140 bytes} [Mon Oct 17 06:49:49 2022] POST /identity/v3/projects => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:49:51.657803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286e764-182e-4783-b09b-cbcf5a786a79 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:51.658291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286e764-182e-4783-b09b-cbcf5a786a79 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:51.658371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286e764-182e-4783-b09b-cbcf5a786a79 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:51.659783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 25/51] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:51.664822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207ae7-119d-4462-a654-db59a915d8eb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:51.665174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207ae7-119d-4462-a654-db59a915d8eb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:51.665379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207ae7-119d-4462-a654-db59a915d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:51.683043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-33207ae7-119d-4462-a654-db59a915d8eb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:51.684787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-33207ae7-119d-4462-a654-db59a915d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:51.687447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:51.750169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 27/52] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:52.018622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286e764-182e-4783-b09b-cbcf5a786a79 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:52.045656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:52.118413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286e764-182e-4783-b09b-cbcf5a786a79 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:52.150425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dfdb4-4437-4121-99dd-7233b144e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:52.151054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dfdb4-4437-4121-99dd-7233b144e22d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:52.151373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dfdb4-4437-4121-99dd-7233b144e22d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:52.151707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dfdb4-4437-4121-99dd-7233b144e22d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:52.153406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dfdb4-4437-4121-99dd-7233b144e22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:52.154867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dfdb4-4437-4121-99dd-7233b144e22d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:52.156451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 26/53] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:52.161813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207ae7-119d-4462-a654-db59a915d8eb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:52.358615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207ae7-119d-4462-a654-db59a915d8eb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:52.391408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d410f3-7561-47f7-9cf1-250ee48087a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:52.392007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d410f3-7561-47f7-9cf1-250ee48087a0 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:52.392290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d410f3-7561-47f7-9cf1-250ee48087a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:52.392565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d410f3-7561-47f7-9cf1-250ee48087a0 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:52.393216 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d410f3-7561-47f7-9cf1-250ee48087a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:52.394682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d410f3-7561-47f7-9cf1-250ee48087a0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:52.410504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:52.421847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 28/54] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:49:52 2022] POST /identity/v3/users => generated 275 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:49:55.549004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66db10-ccdd-4e7d-a200-5a30c5e8562a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:55.549497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66db10-ccdd-4e7d-a200-5a30c5e8562a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:55.549497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66db10-ccdd-4e7d-a200-5a30c5e8562a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:55.550820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 27/55] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:55.563767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f919-6924-4faf-9342-dd265ba630cd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:55.564180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f919-6924-4faf-9342-dd265ba630cd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:55.564385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f919-6924-4faf-9342-dd265ba630cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:55.588196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c256f919-6924-4faf-9342-dd265ba630cd None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:55.589407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c256f919-6924-4faf-9342-dd265ba630cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:55.666212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 29/56] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:55.827045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d5ae7-4b5a-4c37-929a-f234cf93c39d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:55.827045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d5ae7-4b5a-4c37-929a-f234cf93c39d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:55.827045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d5ae7-4b5a-4c37-929a-f234cf93c39d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:55.827045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 28/57] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:55.832665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2ff6-0972-4403-a0dd-96eb43565b9a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:55.832952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2ff6-0972-4403-a0dd-96eb43565b9a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:55.833494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2ff6-0972-4403-a0dd-96eb43565b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:55.851057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc855a90-3ada-4267-b556-27533b2cd73a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:55.851563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc855a90-3ada-4267-b556-27533b2cd73a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:55.851983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc855a90-3ada-4267-b556-27533b2cd73a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:55.853802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 29/58] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:55.859098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70924f-d8e9-415d-b7be-9609b00bb791 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:55.859596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70924f-d8e9-415d-b7be-9609b00bb791 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:55.860047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70924f-d8e9-415d-b7be-9609b00bb791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:55.863102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b16c2ff6-0972-4403-a0dd-96eb43565b9a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:55.864454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b16c2ff6-0972-4403-a0dd-96eb43565b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:55.891695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1b70924f-d8e9-415d-b7be-9609b00bb791 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:55.893821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1b70924f-d8e9-415d-b7be-9609b00bb791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:55.966879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 30/59] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 17 06:49:56.008527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 30/60] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:56.065008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70924f-d8e9-415d-b7be-9609b00bb791 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.098106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:56.103255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd30a24-e036-4255-86bf-4c1f760e97e0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.103549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd30a24-e036-4255-86bf-4c1f760e97e0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.103832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd30a24-e036-4255-86bf-4c1f760e97e0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.105216 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 31/61] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:49:56.117456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce80a0-939b-41bd-b29c-39d725891825 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.117737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce80a0-939b-41bd-b29c-39d725891825 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.118013 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce80a0-939b-41bd-b29c-39d725891825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.149633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-93ce80a0-939b-41bd-b29c-39d725891825 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:56.151629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-93ce80a0-939b-41bd-b29c-39d725891825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:56.230015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70924f-d8e9-415d-b7be-9609b00bb791 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.252495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:56.274287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.274927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.275222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.275515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.285886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 32/62] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:56.286638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:49:56.287245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce80a0-939b-41bd-b29c-39d725891825 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:49:56.291938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:56.292782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:56.293608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 31/63] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:49:56.304350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.403527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce80a0-939b-41bd-b29c-39d725891825 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.438977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.439639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.439922 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.440195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.440673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:56.455236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:49:56.461962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:56.462798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:49:56.463661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:49:56.464223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c223b1-eab4-4544-94c4-a199e44d393a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.467054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 33/64] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:49:56.469407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.502469 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.503135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.503550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.503768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.504434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:56.504789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:56.506181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:56.515046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 32/65] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:56.515870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.535997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:56.599032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2be38-9edd-42a7-927b-18b21e289dc7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.630714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6c1d4-4ba4-4b7c-8bea-5f3b926c2bc0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.634008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62282916-b629-457a-8802-9154332bf06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.634641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62282916-b629-457a-8802-9154332bf06e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.634922 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62282916-b629-457a-8802-9154332bf06e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.635369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62282916-b629-457a-8802-9154332bf06e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.646562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62282916-b629-457a-8802-9154332bf06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-62282916-b629-457a-8802-9154332bf06e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:49:56.651765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:56.653203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 34/66] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:49:56.654368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62282916-b629-457a-8802-9154332bf06e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.670768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.671418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.671705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.671983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.672642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:56.672949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:56.674341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:56.682343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 33/67] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:56.684232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.695162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:56.753756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62282916-b629-457a-8802-9154332bf06e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.760913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:56.782376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d677e-e338-443c-98d3-bb4ccf6c0d36 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.788311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb0bf4-0040-418b-8922-c814d6a6e103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.788941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb0bf4-0040-418b-8922-c814d6a6e103 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.789216 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb0bf4-0040-418b-8922-c814d6a6e103 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.789488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb0bf4-0040-418b-8922-c814d6a6e103 None admin] PATH_INFO: `/v3/users/49f890768f9c4e84be27c82316e89a39` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.804558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb0bf4-0040-418b-8922-c814d6a6e103 None admin] RBAC: Authorizing `identity:get_user(user_id=49f890768f9c4e84be27c82316e89a39)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:56.806129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb0bf4-0040-418b-8922-c814d6a6e103 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:56.807792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 35/68] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/users/49f890768f9c4e84be27c82316e89a39 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:56.808987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3fff-5f14-41f8-9bd1-f00cc9279b8b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.809274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3fff-5f14-41f8-9bd1-f00cc9279b8b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.809551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3fff-5f14-41f8-9bd1-f00cc9279b8b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.811046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 36/69] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:56.811784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3fff-5f14-41f8-9bd1-f00cc9279b8b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.819523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.820159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.820442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.820714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.830910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:49:56.836594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:49:56.837440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:49:56.838212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 34/70] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:49:56.838851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.903343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:56.915710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3fff-5f14-41f8-9bd1-f00cc9279b8b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.936428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820c32-681a-4db0-aa56-e5a4d8cced61 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:56.948371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.948965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.949279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.949556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.950195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:56.950492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:56.951889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:56.959501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 37/71] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:56.960273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:56.975253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc1752-8adf-4dcb-864b-50c5642b2079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:56.975891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1752-8adf-4dcb-864b-50c5642b2079 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.976174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1752-8adf-4dcb-864b-50c5642b2079 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.976453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1752-8adf-4dcb-864b-50c5642b2079 None admin] PATH_INFO: `/v3/users/be01c4fc32bc463e9e081f13d9758f92` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.977583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc1752-8adf-4dcb-864b-50c5642b2079 None admin] RBAC: Authorizing `identity:get_user(user_id=be01c4fc32bc463e9e081f13d9758f92)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:56.979059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc1752-8adf-4dcb-864b-50c5642b2079 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:56.980707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 35/72] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/users/be01c4fc32bc463e9e081f13d9758f92 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:56.981797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c76a1-8b87-457c-93c7-4f37709be902 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:56.982080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c76a1-8b87-457c-93c7-4f37709be902 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:56.982356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c76a1-8b87-457c-93c7-4f37709be902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:56.991276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.000159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-196c76a1-8b87-457c-93c7-4f37709be902 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:57.001434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-196c76a1-8b87-457c-93c7-4f37709be902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:49:57.063487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd116042-7e76-4213-9b77-b98f786f82b7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.065969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 36/73] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:49:57.066596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c76a1-8b87-457c-93c7-4f37709be902 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.099742 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.100381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.100644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.100913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.102028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.103513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.105140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 38/74] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:49:56 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.106468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.135743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.171140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c76a1-8b87-457c-93c7-4f37709be902 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.204442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.205046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.205321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.205591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.206245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:49:57.206545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.207939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.216678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 37/75] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:49:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.217412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.229876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.238021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b91de-0d49-498d-a086-9d9aa3ebc0ed None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.275112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.275906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.276489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.276779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] PATH_INFO: `/v3/users/49f890768f9c4e84be27c82316e89a39` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.278082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] RBAC: Authorizing `identity:get_user(user_id=49f890768f9c4e84be27c82316e89a39)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.279565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.281122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 39/76] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:49:57 2022] GET /identity/v3/users/49f890768f9c4e84be27c82316e89a39 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.284483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.318473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a20c42-58a0-41a3-a1f7-ad6719f57746 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.345728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.353440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.354067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.354345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.354617 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.355796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.357255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.358969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 38/77] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:49:57 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.359749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.388196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06949d-5b96-43d7-aeeb-a33f54698594 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.420591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.421283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.421509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.421779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/49f890768f9c4e84be27c82316e89a39/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.423654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] RBAC: Authorizing `identity:create_grant(project_id=66ff1cdc507747e5a54adbea719b6323, user_id=49f890768f9c4e84be27c82316e89a39, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.425081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.437271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 40/78] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:49:57 2022] PUT /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/49f890768f9c4e84be27c82316e89a39/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:49:57.438140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.471729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.483233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c808eb-0a9a-4c5f-9b94-7b329cb3ae10 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.521296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.521938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.522219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.522491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] PATH_INFO: `/v3/users/49f890768f9c4e84be27c82316e89a39` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.523687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] RBAC: Authorizing `identity:get_user(user_id=49f890768f9c4e84be27c82316e89a39)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.525191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.532518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 39/79] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:49:57 2022] GET /identity/v3/users/49f890768f9c4e84be27c82316e89a39 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.532518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.569389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e481-2128-4570-ac57-bbab3b5a3c32 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.600507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.608350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.609687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.609687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.609687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.611126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.612361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.614898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 41/80] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:49:57 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.615700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.651378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4e0dd-c73f-4976-a481-d665b184b38c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.686607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.687161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.687468 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.687744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.689094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.690549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.692134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 40/81] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:49:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.692968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.715603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.727142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea43663-b294-4c31-be88-dcf39caed448 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.761865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.762525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.762801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.763074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/be01c4fc32bc463e9e081f13d9758f92/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.764980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=66ff1cdc507747e5a54adbea719b6323, user_id=be01c4fc32bc463e9e081f13d9758f92, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.766439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.781616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 42/82] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:49:57 2022] PUT /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/be01c4fc32bc463e9e081f13d9758f92/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:49:57.782414 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.823674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69406146-4e05-4363-8176-c61e7b0119c9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.869698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:57.877209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.878854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.879395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.879882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] PATH_INFO: `/v3/projects/ea6a4787e9394ee4bedf9135f32b6122` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.881949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] RBAC: Authorizing `identity:get_project(project_id=ea6a4787e9394ee4bedf9135f32b6122)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.884436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.887125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 41/83] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:49:57 2022] GET /identity/v3/projects/ea6a4787e9394ee4bedf9135f32b6122 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:49:57.888704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:57.905735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449a87d-9cfc-43f4-9ff1-7c2898b584e5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:57.951431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:57.952203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:57.952543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:57.952894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/49f890768f9c4e84be27c82316e89a39/roles/ca9ae3dc2d9b42c0b35eca1fda566496` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:57.955267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] RBAC: Authorizing `identity:create_grant(project_id=66ff1cdc507747e5a54adbea719b6323, user_id=49f890768f9c4e84be27c82316e89a39, role_id=ca9ae3dc2d9b42c0b35eca1fda566496)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:57.957094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:57.973443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 43/84] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:49:57 2022] PUT /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/49f890768f9c4e84be27c82316e89a39/roles/ca9ae3dc2d9b42c0b35eca1fda566496 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:49:57.980075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:49:58.000966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:49:58.036289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd564a-eb8e-4ba1-88b3-4c3780ebc32a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:58.087612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf9400b-274d-4ee5-80c3-871dfbf7764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:58.089581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9400b-274d-4ee5-80c3-871dfbf7764d None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:58.090080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9400b-274d-4ee5-80c3-871dfbf7764d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:58.090532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9400b-274d-4ee5-80c3-871dfbf7764d None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:58.091460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9400b-274d-4ee5-80c3-871dfbf7764d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:58.093099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9400b-274d-4ee5-80c3-871dfbf7764d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:58.126497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d24ff-a1e8-49fa-86cc-ed3987645f39 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:49:58.130460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 42/85] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:49:57 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:49:58.170367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd957c-cfc9-46b7-8e86-cb6fe1120bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:49:58.171046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd957c-cfc9-46b7-8e86-cb6fe1120bc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:58.171443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd957c-cfc9-46b7-8e86-cb6fe1120bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:58.171720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd957c-cfc9-46b7-8e86-cb6fe1120bc5 None admin] PATH_INFO: `/v3/projects/ea6a4787e9394ee4bedf9135f32b6122/users/49f890768f9c4e84be27c82316e89a39/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:58.173985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd957c-cfc9-46b7-8e86-cb6fe1120bc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea6a4787e9394ee4bedf9135f32b6122, user_id=49f890768f9c4e84be27c82316e89a39, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:49:58.175636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd957c-cfc9-46b7-8e86-cb6fe1120bc5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:49:58.191028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 44/86] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:49:57 2022] PUT /identity/v3/projects/ea6a4787e9394ee4bedf9135f32b6122/users/49f890768f9c4e84be27c82316e89a39/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:49:59.896370 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f16ee-c6c1-414a-a623-94727771394c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:59.896909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f16ee-c6c1-414a-a623-94727771394c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:59.896909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f16ee-c6c1-414a-a623-94727771394c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:59.898402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 43/87] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:49:59.912850 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445987ab-3a09-48b2-8c51-a8280684170d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:49:59.913335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445987ab-3a09-48b2-8c51-a8280684170d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:49:59.913409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445987ab-3a09-48b2-8c51-a8280684170d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:49:59.936721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-445987ab-3a09-48b2-8c51-a8280684170d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:49:59.937905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-445987ab-3a09-48b2-8c51-a8280684170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:00.010291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:00.015645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 45/88] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:00.298585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f16ee-c6c1-414a-a623-94727771394c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:00.368980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:00.420361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f16ee-c6c1-414a-a623-94727771394c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:00.455477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f6ef8-06ba-407d-8bde-238aa3e75e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:00.456162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f6ef8-06ba-407d-8bde-238aa3e75e83 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:00.456444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f6ef8-06ba-407d-8bde-238aa3e75e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:00.456714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f6ef8-06ba-407d-8bde-238aa3e75e83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:00.468783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f6ef8-06ba-407d-8bde-238aa3e75e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-599f6ef8-06ba-407d-8bde-238aa3e75e83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:00.473491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:00.474411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:00.475178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 44/89] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:00.481764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445987ab-3a09-48b2-8c51-a8280684170d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:00.499961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225137-20b6-41b8-97ba-57c3b854a721 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:00.500238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225137-20b6-41b8-97ba-57c3b854a721 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:00.500505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225137-20b6-41b8-97ba-57c3b854a721 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:00.501836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 45/90] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:00.507152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54e1eb-d813-412b-af63-f55560fa271b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:00.507457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54e1eb-d813-412b-af63-f55560fa271b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:00.507732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54e1eb-d813-412b-af63-f55560fa271b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:00.535153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5b54e1eb-d813-412b-af63-f55560fa271b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:00.536362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b54e1eb-d813-412b-af63-f55560fa271b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:00.639297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445987ab-3a09-48b2-8c51-a8280684170d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:00.652629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 46/91] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:00.691613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a3c98-3030-42e9-9eea-8706c452301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:00.692149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a3c98-3030-42e9-9eea-8706c452301d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:00.692435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a3c98-3030-42e9-9eea-8706c452301d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:00.692716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a3c98-3030-42e9-9eea-8706c452301d None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:00.693382 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a3c98-3030-42e9-9eea-8706c452301d None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:00.693686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a3c98-3030-42e9-9eea-8706c452301d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:00.695066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a3c98-3030-42e9-9eea-8706c452301d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:00.707819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 46/92] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:00.713604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54e1eb-d813-412b-af63-f55560fa271b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:00.733199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dce26d-637f-4c1c-b181-d8c8157ab7bb None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:00.734056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dce26d-637f-4c1c-b181-d8c8157ab7bb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:00.734561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dce26d-637f-4c1c-b181-d8c8157ab7bb None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:00.736132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 47/93] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:00.741763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8061256-c6fe-47fc-8a1a-ec442a9ee717 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:00.742264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8061256-c6fe-47fc-8a1a-ec442a9ee717 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:00.742768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8061256-c6fe-47fc-8a1a-ec442a9ee717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:00.782452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e8061256-c6fe-47fc-8a1a-ec442a9ee717 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:00.783664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e8061256-c6fe-47fc-8a1a-ec442a9ee717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:00.799929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:00.884815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:00.899885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54e1eb-d813-412b-af63-f55560fa271b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:00.905349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 48/94] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:00.946578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:00.947138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:00.947436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:00.947707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] PATH_INFO: `/v3/users/49f890768f9c4e84be27c82316e89a39` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:00.948819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] RBAC: Authorizing `identity:get_user(user_id=49f890768f9c4e84be27c82316e89a39)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:00.950806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:00.952390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 47/95] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:00 2022] GET /identity/v3/users/49f890768f9c4e84be27c82316e89a39 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:00.958052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8061256-c6fe-47fc-8a1a-ec442a9ee717 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.066176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.098647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8061256-c6fe-47fc-8a1a-ec442a9ee717 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.163079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.169421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.169756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.170030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.262751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb8cfd-d5f3-48da-849a-b16aeeba47d2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.274329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:01.279053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:01.279053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:01.287307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:01.310405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.311033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.311381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.311660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.322554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:01.328044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:01.328869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:01.329632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 48/96] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:01.330246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.462442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9469ea9-6ec7-4cda-9773-d57480e9b1aa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.508991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 49/97] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:01.509843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.553240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.553821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.554102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.554378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.588174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:01.594029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:01.594963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:01.595756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 49/98] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:50:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:01.596397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.614679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:01.626081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc251327-bd76-4477-ac6f-833d10b801a4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.663236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:01.664376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.665287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.665572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.666527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.667196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:01.667516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:01.669010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:01.678486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 50/99] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:01.680212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.741414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f922f-ecbf-4f38-8fcd-7d63491ad3ed None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.777024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cfca9-ad91-4c32-bcdd-1868fa15da84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.778014 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cfca9-ad91-4c32-bcdd-1868fa15da84 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.778497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cfca9-ad91-4c32-bcdd-1868fa15da84 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.778972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cfca9-ad91-4c32-bcdd-1868fa15da84 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.780451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cfca9-ad91-4c32-bcdd-1868fa15da84 None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:01.782225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cfca9-ad91-4c32-bcdd-1868fa15da84 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:01.784113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 50/100] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:01 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:01.785524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb6d86-903d-4db2-acb1-21f78ea89d45 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.786066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb6d86-903d-4db2-acb1-21f78ea89d45 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.786467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df26d3-bc3c-4302-8ceb-880a2707c3cb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.786741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb6d86-903d-4db2-acb1-21f78ea89d45 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.788458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 51/101] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:01.789632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb6d86-903d-4db2-acb1-21f78ea89d45 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.827044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.827699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.827982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.828257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.828907 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:01.829991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:01.830664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:01.839692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 51/102] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:01.840512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:01.867650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:01.905013 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb6d86-903d-4db2-acb1-21f78ea89d45 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.907575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:01.943171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e43bb-1a58-4ba0-b740-33242899dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.944164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e43bb-1a58-4ba0-b740-33242899dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.944664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e43bb-1a58-4ba0-b740-33242899dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.945139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e43bb-1a58-4ba0-b740-33242899dfc2 None admin] PATH_INFO: `/v3/users/be01c4fc32bc463e9e081f13d9758f92` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.946506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e43bb-1a58-4ba0-b740-33242899dfc2 None admin] RBAC: Authorizing `identity:get_user(user_id=be01c4fc32bc463e9e081f13d9758f92)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:01.948335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e43bb-1a58-4ba0-b740-33242899dfc2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:01.950736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 52/103] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:01 2022] GET /identity/v3/users/be01c4fc32bc463e9e081f13d9758f92 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:01.952081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591c2c1-2ea6-4ea6-b609-6195be5048f8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.952320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2d2b7-1e1a-4fe6-a68e-10d2d95f5a68 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:01.952802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591c2c1-2ea6-4ea6-b609-6195be5048f8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.953270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591c2c1-2ea6-4ea6-b609-6195be5048f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.973186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4591c2c1-2ea6-4ea6-b609-6195be5048f8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:01.974840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4591c2c1-2ea6-4ea6-b609-6195be5048f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:01.990985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:01.991774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:01.992014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:01.992405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:01.993148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] RBAC: Adding query filter params (role.id=84eb2fe76ba84dfab84d2d1ab664754f, scope.project.id=66ff1cdc507747e5a54adbea719b6323, user.id=49f890768f9c4e84be27c82316e89a39) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:01.993459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:01.994930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:02.011265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 52/104] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:01 2022] GET /identity/v3/role_assignments?user.id=49f890768f9c4e84be27c82316e89a39&scope.project.id=66ff1cdc507747e5a54adbea719b6323&role.id=84eb2fe76ba84dfab84d2d1ab664754f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:02.011265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.067255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 53/105] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:02.068566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591c2c1-2ea6-4ea6-b609-6195be5048f8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.114539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.137380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b0cfd-4f8e-410a-8108-41fac7642470 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:02.174990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ae05c-8f40-4315-b495-91635bd18ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:02.175670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ae05c-8f40-4315-b495-91635bd18ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.175946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ae05c-8f40-4315-b495-91635bd18ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.176219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ae05c-8f40-4315-b495-91635bd18ed3 None admin] PATH_INFO: `/v3/users/49f890768f9c4e84be27c82316e89a39` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.177355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ae05c-8f40-4315-b495-91635bd18ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=49f890768f9c4e84be27c82316e89a39)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:02.179008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ae05c-8f40-4315-b495-91635bd18ed3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:02.180684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 53/106] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/users/49f890768f9c4e84be27c82316e89a39 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:02.181944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39614ce9-ed71-4e3a-8702-293581f02b98 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.182264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39614ce9-ed71-4e3a-8702-293581f02b98 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.182540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39614ce9-ed71-4e3a-8702-293581f02b98 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.184419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 54/107] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:02.185428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb4b8-776d-4be1-8e71-4acbb3671eb5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.185702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb4b8-776d-4be1-8e71-4acbb3671eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.185979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb4b8-776d-4be1-8e71-4acbb3671eb5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.187428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 55/108] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:02.192943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c622b-a4c8-4909-8c18-4f01baad304b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.193231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c622b-a4c8-4909-8c18-4f01baad304b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.193502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c622b-a4c8-4909-8c18-4f01baad304b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.202708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.206350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591c2c1-2ea6-4ea6-b609-6195be5048f8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:02.215378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f86c622b-a4c8-4909-8c18-4f01baad304b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:02.216452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f86c622b-a4c8-4909-8c18-4f01baad304b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:02.254982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:02.255618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.255897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.256168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.298624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 56/109] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:02.299979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5eafc-b729-43b3-b088-f50c8a35bebb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.300273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5eafc-b729-43b3-b088-f50c8a35bebb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.300551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5eafc-b729-43b3-b088-f50c8a35bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.320135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1aa5eafc-b729-43b3-b088-f50c8a35bebb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:02.321322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1aa5eafc-b729-43b3-b088-f50c8a35bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:02.347824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:02.349752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:02.391958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 57/110] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:02.393036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5eafc-b729-43b3-b088-f50c8a35bebb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.402913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.464352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.474772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 54/111] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:02.475848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.505877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5eafc-b729-43b3-b088-f50c8a35bebb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:02.545951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:02.546794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.547230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.547652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.606301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250905c0-eb39-4b22-82e1-905548cdfd0c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:02.619552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:02.621121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:02.630739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.644817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:02.645374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.645651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.645945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.656813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:02.662724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:02.663608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 55/112] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:02.670088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.722183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 58/113] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:02.723026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.729118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.769113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50137f6a-f360-4f11-bac0-d4b7fdd7781c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:02.801964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:02.802582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.802858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.803130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.804289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:02.805741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:02.807372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 56/114] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:02.808120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.820511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51e2c-ab4e-42be-ae52-d81fa9489199 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:02.829570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.858788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:02.859441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.859721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.859990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.869762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:02.874986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:02.876041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 59/115] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:02.876971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.915873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98481d71-7046-41d2-93f6-fa71fe00fd87 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:02.941554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:02.949572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:02.950181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:02.950463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:02.951079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:02.951423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:02.951728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:02.953111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:02.961088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 57/116] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:02.961857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:02.978282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c95447-04c9-4968-a2da-b2496d575f76 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.011900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.012526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.012800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.013073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.014173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.015667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.017217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 60/117] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:03.018054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.051157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.059003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0c138-82f6-4a33-bb9d-82339bc182cd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.092189 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.092816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.093104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.093388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.094503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.095980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.097599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 58/118] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:02 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:03.098356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.115435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88397-af6f-4b76-8aff-9652555ed77a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.148919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.149513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.149788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.150058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.150772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=66ff1cdc507747e5a54adbea719b6323, user.id=be01c4fc32bc463e9e081f13d9758f92) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:03.151073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.153086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.155457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.160256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 61/119] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/role_assignments?user.id=be01c4fc32bc463e9e081f13d9758f92&scope.project.id=66ff1cdc507747e5a54adbea719b6323&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:03.161089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.213773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa8468-35a1-489c-8689-dab958f88b5a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.249721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.250400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.250683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.250956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.251666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:03.251972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.253405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.263095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 59/120] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:03.263787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.281898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.286442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63741681-1085-4e14-8228-6f039c83eabf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.324536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.325282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.325565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.325839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] PATH_INFO: `/v3/users/49f890768f9c4e84be27c82316e89a39` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.326993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] RBAC: Authorizing `identity:get_user(user_id=49f890768f9c4e84be27c82316e89a39)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.328499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.330096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 62/121] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/users/49f890768f9c4e84be27c82316e89a39 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:03.335591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.377102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fe7f9-9fb7-456f-89d1-85759439addb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.400042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.414566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.415168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.415465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.415732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.416369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.418335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.443762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460e262-e58e-4c57-8f18-a007ac3628e2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.445420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 60/122] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:50:03 2022] POST /identity/v3/projects => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:03.446220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.479499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.480550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.480830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.481102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.481855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] RBAC: Adding query filter params (role.id=ca9ae3dc2d9b42c0b35eca1fda566496, scope.project.id=66ff1cdc507747e5a54adbea719b6323, user.id=49f890768f9c4e84be27c82316e89a39) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:03.482137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.483646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.489934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 63/123] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/role_assignments?user.id=49f890768f9c4e84be27c82316e89a39&scope.project.id=66ff1cdc507747e5a54adbea719b6323&role.id=ca9ae3dc2d9b42c0b35eca1fda566496 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:03.496478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.539443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.581621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420cc04-06cd-4a55-80f2-6848e9cb790a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.625869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.627004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.627604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.628121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] PATH_INFO: `/v3/users/1ef61036957645ab9d4d8a08f1632ead` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.636015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b34f-f241-4376-ab51-ec7f888af5a3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.646431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] RBAC: Authorizing `identity:get_user(user_id=1ef61036957645ab9d4d8a08f1632ead)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.648090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.650131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 61/124] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/users/1ef61036957645ab9d4d8a08f1632ead => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:03.656257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.686281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.686281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.686559 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.686785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.693015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.777519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ccf6d-0af0-406e-bdf9-41cf9e7739a9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.786150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.788112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.812637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.813515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.813794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.814069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.815167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.816637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.818463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 62/125] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:03.820363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.824091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.892604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 64/126] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:03.898065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:03.921988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:03.926751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57065ee1-e855-489b-9036-d39c90dcfd2c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:03.963479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:03.964135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:03.964988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:03.965232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/1ef61036957645ab9d4d8a08f1632ead/roles/502d111dad8c4856b928ecc094b02d26` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:03.967147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] RBAC: Authorizing `identity:create_grant(project_id=66ff1cdc507747e5a54adbea719b6323, user_id=1ef61036957645ab9d4d8a08f1632ead, role_id=502d111dad8c4856b928ecc094b02d26)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:03.968616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:03.983116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 63/127] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:03 2022] PUT /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323/users/1ef61036957645ab9d4d8a08f1632ead/roles/502d111dad8c4856b928ecc094b02d26 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:04.001596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b458485-676f-478f-9ce0-5514f1d4e480 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:04.035105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:04.042296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc6dc1-2128-4232-9cb4-11fa8e2cb4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:04.043566 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc6dc1-2128-4232-9cb4-11fa8e2cb4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:04.043848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc6dc1-2128-4232-9cb4-11fa8e2cb4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:04.044118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc6dc1-2128-4232-9cb4-11fa8e2cb4a7 None admin] PATH_INFO: `/v3/projects/ea6a4787e9394ee4bedf9135f32b6122` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:04.045332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc6dc1-2128-4232-9cb4-11fa8e2cb4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ea6a4787e9394ee4bedf9135f32b6122)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:04.046845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc6dc1-2128-4232-9cb4-11fa8e2cb4a7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:04.048782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 65/128] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:03 2022] GET /identity/v3/projects/ea6a4787e9394ee4bedf9135f32b6122 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:04.054714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:04.175297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2ac67-d508-4ab7-b5a7-347380cc73be None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:04.209589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e6c3e-92d0-4243-970c-19365e51c453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:04.210201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6c3e-92d0-4243-970c-19365e51c453 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:04.210477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6c3e-92d0-4243-970c-19365e51c453 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:04.210745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6c3e-92d0-4243-970c-19365e51c453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:04.211501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e6c3e-92d0-4243-970c-19365e51c453 None admin] RBAC: Adding query filter params (role.id=84eb2fe76ba84dfab84d2d1ab664754f, scope.project.id=ea6a4787e9394ee4bedf9135f32b6122, user.id=49f890768f9c4e84be27c82316e89a39) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:04.211809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e6c3e-92d0-4243-970c-19365e51c453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:04.213248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e6c3e-92d0-4243-970c-19365e51c453 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:04.221151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 64/129] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:04 2022] GET /identity/v3/role_assignments?user.id=49f890768f9c4e84be27c82316e89a39&scope.project.id=ea6a4787e9394ee4bedf9135f32b6122&role.id=84eb2fe76ba84dfab84d2d1ab664754f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:05.691115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692346a7-bf31-4139-a1a2-f1fa494b149a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:05.691652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692346a7-bf31-4139-a1a2-f1fa494b149a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:05.691734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692346a7-bf31-4139-a1a2-f1fa494b149a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:05.693070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 66/130] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:05.698402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d955f9-7ed1-43fd-a0fe-39cdf4cc31f2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:05.699388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d955f9-7ed1-43fd-a0fe-39cdf4cc31f2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:05.699475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d955f9-7ed1-43fd-a0fe-39cdf4cc31f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:05.717872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-62d955f9-7ed1-43fd-a0fe-39cdf4cc31f2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:05.719067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-62d955f9-7ed1-43fd-a0fe-39cdf4cc31f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:05.735175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:05.785606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 65/131] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:06.051651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692346a7-bf31-4139-a1a2-f1fa494b149a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:06.116681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c04f-54c3-4d51-be12-2aed987db1a6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:06.116966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c04f-54c3-4d51-be12-2aed987db1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:06.117247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c04f-54c3-4d51-be12-2aed987db1a6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:06.118582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 66/132] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:06.123572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27775c4-a45d-47c7-a352-df2346dd7be3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:06.123861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27775c4-a45d-47c7-a352-df2346dd7be3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:06.124144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27775c4-a45d-47c7-a352-df2346dd7be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:06.141539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f27775c4-a45d-47c7-a352-df2346dd7be3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:06.142721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f27775c4-a45d-47c7-a352-df2346dd7be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:06.153161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692346a7-bf31-4139-a1a2-f1fa494b149a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:06.187529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:06.188405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:06.188685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:06.188962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:06.190049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:06.191531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:06.193149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 67/133] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:06.199261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:06.210968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 67/134] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:06.263760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:06.297344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76932106-a5ce-470c-8cc1-6b5efb7312b4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:06.331330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:06.331871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:06.332151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:06.332425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:06.333062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:06.334498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:06.354332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 68/135] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:06 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:50:06.476464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27775c4-a45d-47c7-a352-df2346dd7be3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:06.560172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:06.578598 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27775c4-a45d-47c7-a352-df2346dd7be3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:06.613350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:06.613916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:06.614195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:06.614471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:06.626200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:06.631186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:06.632132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:06.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 68/136] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:06.638305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:06.747712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc0d8b-4c71-4f1f-ae54-f420f2d87e07 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:06.772198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:06.786487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:06.787114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:06.787430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:06.787710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:06.788384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:06.788692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:06.790076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:06.799360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 69/137] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:06.805104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:06.916618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aa45c-b6be-49e7-aa8e-db362b3f9523 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:06.953040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:06.953894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:06.954311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:06.954761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] PATH_INFO: `/v3/users/1ef61036957645ab9d4d8a08f1632ead` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:06.956086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] RBAC: Authorizing `identity:get_user(user_id=1ef61036957645ab9d4d8a08f1632ead)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:06.957678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:06.959411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 69/138] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:06 2022] GET /identity/v3/users/1ef61036957645ab9d4d8a08f1632ead => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:06.967261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:07.068909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8941a1-ee05-45f6-a578-bb3b6c96b27a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:07.103344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:07.103960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:07.104246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:07.104527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:07.171877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:07.175751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:07.177366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:07.284554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 70/139] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:07.285901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:07.300544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:07.395584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97b9ee-c9b9-450e-83d1-2c6b734e2198 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:07.430755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb5001-47e7-4347-975d-609e86e4f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:07.431332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5001-47e7-4347-975d-609e86e4f821 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:07.431668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5001-47e7-4347-975d-609e86e4f821 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:07.431945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5001-47e7-4347-975d-609e86e4f821 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:07.433846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb5001-47e7-4347-975d-609e86e4f821 None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:07.435311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb5001-47e7-4347-975d-609e86e4f821 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:07.437205 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 70/140] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:07 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:07.443123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:07.526138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:07.542649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc903f5-4616-45f0-b330-629c98fbe532 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:07.578805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6891e-d10d-4f68-9ac9-6190fdd38148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:07.579434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6891e-d10d-4f68-9ac9-6190fdd38148 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:07.579723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6891e-d10d-4f68-9ac9-6190fdd38148 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:07.579990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6891e-d10d-4f68-9ac9-6190fdd38148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:07.580696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6891e-d10d-4f68-9ac9-6190fdd38148 None admin] RBAC: Adding query filter params (role.id=502d111dad8c4856b928ecc094b02d26, scope.project.id=66ff1cdc507747e5a54adbea719b6323, user.id=1ef61036957645ab9d4d8a08f1632ead) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:07.580993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6891e-d10d-4f68-9ac9-6190fdd38148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:07.582423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6891e-d10d-4f68-9ac9-6190fdd38148 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:07.588346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 71/141] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:07 2022] GET /identity/v3/role_assignments?user.id=1ef61036957645ab9d4d8a08f1632ead&scope.project.id=66ff1cdc507747e5a54adbea719b6323&role.id=502d111dad8c4856b928ecc094b02d26 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:08.597901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376280e3-24af-49ee-b364-56a39fc459b2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:08.598379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376280e3-24af-49ee-b364-56a39fc459b2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:08.598379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376280e3-24af-49ee-b364-56a39fc459b2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:08.599773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 71/142] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:08.605289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657bb19-4e3c-4a2c-b32d-4d48f552d4a3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:08.605685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657bb19-4e3c-4a2c-b32d-4d48f552d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:08.605838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657bb19-4e3c-4a2c-b32d-4d48f552d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:08.623930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1657bb19-4e3c-4a2c-b32d-4d48f552d4a3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:08.625080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1657bb19-4e3c-4a2c-b32d-4d48f552d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:08.692413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 72/143] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:08.830272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317825e6-d250-4a9b-a2bb-f25dcb080f70 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:08.830555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317825e6-d250-4a9b-a2bb-f25dcb080f70 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:08.830997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317825e6-d250-4a9b-a2bb-f25dcb080f70 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:08.832516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 72/144] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:08.837569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d6fe-af59-453a-8d9a-400a1ae8f37f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:08.837855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d6fe-af59-453a-8d9a-400a1ae8f37f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:08.838137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d6fe-af59-453a-8d9a-400a1ae8f37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:08.857299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9df6d6fe-af59-453a-8d9a-400a1ae8f37f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:08.858407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9df6d6fe-af59-453a-8d9a-400a1ae8f37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:08.872791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03684e-ac94-466c-a58c-825ae1af79de None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:08.873078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03684e-ac94-466c-a58c-825ae1af79de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:08.873362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03684e-ac94-466c-a58c-825ae1af79de None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:08.874687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 73/145] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:08.879505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4995fe-ecc3-4a26-8032-e9f1d4c543a4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:08.879934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4995fe-ecc3-4a26-8032-e9f1d4c543a4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:08.880409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4995fe-ecc3-4a26-8032-e9f1d4c543a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:08.898158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6d4995fe-ecc3-4a26-8032-e9f1d4c543a4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:08.899594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6d4995fe-ecc3-4a26-8032-e9f1d4c543a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:08.927439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 73/146] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:08.958904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6d6fe-af59-453a-8d9a-400a1ae8f37f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:08.971849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:08.972393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 74/147] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:09.057398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6d6fe-af59-453a-8d9a-400a1ae8f37f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.090521 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.091142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.091443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.091710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.101518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:09.106725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:09.107824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:09.108591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:09.111522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 74/148] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:09.113488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4995fe-ecc3-4a26-8032-e9f1d4c543a4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.123960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:09.196479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.220336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4995fe-ecc3-4a26-8032-e9f1d4c543a4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.253921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.254520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.254791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.255057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.255742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:09.256039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.257408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.265457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 75/149] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:09.266243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.270503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:09.301394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195c15-2bc0-4283-b7dd-d2ae75d44c2f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.333236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:09.335893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.336542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.336817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.337087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.347633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:09.352303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:09.353161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:09.353952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 75/150] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:09.355187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.367516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d987f66-800c-4cd7-81e6-541b0b943553 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.400096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.400714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.400992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.401262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.403195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.404680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.406227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 76/151] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:09.411336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.454342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79171c61-33e5-4087-966c-e12265538e55 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.483271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:09.489352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.490000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.490274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.490545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] PATH_INFO: `/v3/users/81f3cbebf5174585af562fe7c97475ad` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.504679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] RBAC: Authorizing `identity:get_user(user_id=81f3cbebf5174585af562fe7c97475ad)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.506232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.507959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 76/152] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/users/81f3cbebf5174585af562fe7c97475ad => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:09.508778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.513520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f482cf8-062d-4aa4-b5ed-573509152744 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.546921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.547571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.547848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.548123 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.548778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:09.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.550466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.555326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:09.563366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 77/153] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:09.564359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.607775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d55a5-ad99-4a7f-bbbb-1d6514e35b72 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.640864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.641400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.641686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.641955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.642592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.645601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.660348 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c2134-eb2c-462f-903a-2964d5255f71 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.666205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 77/154] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:09 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:09.666931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.708936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.709881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.710160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.710435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.711614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.713084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.714826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 78/155] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:09.720793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.723270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:09.789209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e533d2-4808-4e64-83c4-9468d9d0e3fc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.797662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:09.827933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.828572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.828852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.829111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] PATH_INFO: `/v3/users/81f3cbebf5174585af562fe7c97475ad` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.830256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] RBAC: Authorizing `identity:get_user(user_id=81f3cbebf5174585af562fe7c97475ad)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.831752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.833313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 78/156] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/users/81f3cbebf5174585af562fe7c97475ad => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:09.839798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:09.841104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfb4bc-fef5-4939-ad4a-26e219f7f764 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.879116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:09.879822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:09.880104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:09.880386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db/users/81f3cbebf5174585af562fe7c97475ad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:09.882327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6d647839334404c995bca069f96e0db, user_id=81f3cbebf5174585af562fe7c97475ad, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:09.883830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:09.898143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 79/157] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:09 2022] PUT /identity/v3/projects/a6d647839334404c995bca069f96e0db/users/81f3cbebf5174585af562fe7c97475ad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:50:09.968258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7c528-8ca1-4795-b8e4-f81336db09c0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:09.986128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:10.011679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0082866-806d-4e61-a5a4-f4fe5c5bb569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:10.012602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0082866-806d-4e61-a5a4-f4fe5c5bb569 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:10.013036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0082866-806d-4e61-a5a4-f4fe5c5bb569 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:10.013447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0082866-806d-4e61-a5a4-f4fe5c5bb569 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:10.014753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0082866-806d-4e61-a5a4-f4fe5c5bb569 None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:10.016391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0082866-806d-4e61-a5a4-f4fe5c5bb569 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:10.018267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 79/158] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:09 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:10.024366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:10.139571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b755d7-4a4c-46d8-a6fc-752b6cd40d20 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:10.174461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:10.182811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db03dc4-af30-4419-af1a-2513f74fd96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:10.183503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db03dc4-af30-4419-af1a-2513f74fd96d None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:10.183780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db03dc4-af30-4419-af1a-2513f74fd96d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:10.184053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db03dc4-af30-4419-af1a-2513f74fd96d None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db/users/81f3cbebf5174585af562fe7c97475ad/roles/ca9ae3dc2d9b42c0b35eca1fda566496` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:10.186438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db03dc4-af30-4419-af1a-2513f74fd96d None admin] RBAC: Authorizing `identity:create_grant(project_id=a6d647839334404c995bca069f96e0db, user_id=81f3cbebf5174585af562fe7c97475ad, role_id=ca9ae3dc2d9b42c0b35eca1fda566496)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:10.188251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db03dc4-af30-4419-af1a-2513f74fd96d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:10.203144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 80/159] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:10 2022] PUT /identity/v3/projects/a6d647839334404c995bca069f96e0db/users/81f3cbebf5174585af562fe7c97475ad/roles/ca9ae3dc2d9b42c0b35eca1fda566496 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:11.963075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f6a1-27d3-49e9-b39d-29229e7777ea None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:11.963691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f6a1-27d3-49e9-b39d-29229e7777ea None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:11.963691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f6a1-27d3-49e9-b39d-29229e7777ea None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:11.965819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 80/160] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:11.971631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075556bd-1f1d-4824-b88f-8d5378a8a157 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:11.972172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075556bd-1f1d-4824-b88f-8d5378a8a157 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:11.972172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075556bd-1f1d-4824-b88f-8d5378a8a157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:11.991616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-075556bd-1f1d-4824-b88f-8d5378a8a157 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:11.992808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-075556bd-1f1d-4824-b88f-8d5378a8a157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:12.068180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 81/161] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:12.082129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0a7ab-a9a7-4dbe-8b27-f0fa6d5561ce None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.082408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0a7ab-a9a7-4dbe-8b27-f0fa6d5561ce None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.082681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0a7ab-a9a7-4dbe-8b27-f0fa6d5561ce None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.084330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 81/162] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:12.090077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8cac8-296a-4c69-8916-2c7083387650 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.090357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8cac8-296a-4c69-8916-2c7083387650 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.090630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8cac8-296a-4c69-8916-2c7083387650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.109945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a3e8cac8-296a-4c69-8916-2c7083387650 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:12.111115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a3e8cac8-296a-4c69-8916-2c7083387650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:12.134411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ee9d-cb35-43a6-9eee-db82db9ed42d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.134685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ee9d-cb35-43a6-9eee-db82db9ed42d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.134959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ee9d-cb35-43a6-9eee-db82db9ed42d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.136682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 82/163] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:12.141510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd301-6eb8-4df2-a07d-d8f2365d3f98 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.141804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd301-6eb8-4df2-a07d-d8f2365d3f98 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.142113 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd301-6eb8-4df2-a07d-d8f2365d3f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.172089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7b2bd301-6eb8-4df2-a07d-d8f2365d3f98 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:12.173642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7b2bd301-6eb8-4df2-a07d-d8f2365d3f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:12.184306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 82/164] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:12.244422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 83/165] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:12.340116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8cac8-296a-4c69-8916-2c7083387650 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.375355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b625baa-fd15-48de-914e-9552565b244f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.375681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b625baa-fd15-48de-914e-9552565b244f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.375915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b625baa-fd15-48de-914e-9552565b244f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.377267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 84/166] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:12.382351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3863d0-045a-49f8-b0cb-53a17ae11e4a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.382621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3863d0-045a-49f8-b0cb-53a17ae11e4a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.382893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3863d0-045a-49f8-b0cb-53a17ae11e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.391676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:12.402848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7f3863d0-045a-49f8-b0cb-53a17ae11e4a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:12.404543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7f3863d0-045a-49f8-b0cb-53a17ae11e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:12.451771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8cac8-296a-4c69-8916-2c7083387650 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:12.484930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 85/167] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:12.486050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3863d0-045a-49f8-b0cb-53a17ae11e4a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.497596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:12.498966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.499579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.500064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.514940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:12.522233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:12.523152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:12.524079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 83/168] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:12.524761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.532567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:12.593934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3863d0-045a-49f8-b0cb-53a17ae11e4a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:12.629073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:12.629692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.629967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.630233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.631348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:12.631446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64317d5f-ec62-4d80-9daa-cc64e6e003dc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:12.632802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:12.634514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 86/169] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:12.635100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.645904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:12.671795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:12.672412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.672685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.672949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.683635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:12.688569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:12.689441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:12.690240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 84/170] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:50:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:12.690895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.750729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb36492-c3fa-4996-b106-3b1c1fa2a5ea None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:12.760313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:12.785154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:12.785755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.786030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.786293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.786933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:12.787247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:12.788612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:12.795016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c5152-f804-49bf-b105-d9283962ad47 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:12.796829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 87/171] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:12.797586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.834613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:12.835237 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.835506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.835772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.836403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:12.837864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:12.858408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 85/172] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:12 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:12.859250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.889158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:12.913990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6e807-c37b-496f-9a3e-558ec87ebec5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:12.954843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:12.955542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:12.955815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:12.956082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:12.956743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:12.957059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:12.958440 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:12.967540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 88/173] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:12.968406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:12.984165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d89d6-42dd-4029-bb8c-c301eda8c3cd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.002607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.023946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.024586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.024864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.025132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.037450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:13.043597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:13.044495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:13.045261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 86/174] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:50:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:13.045925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.092473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d9d92-ab0b-4c61-a246-5915af4f7ae1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.142718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.143475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.143758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.144029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] PATH_INFO: `/v3/users/c86cb163d2dd445c8792640a1d5200cf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.148351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.159227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] RBAC: Authorizing `identity:get_user(user_id=c86cb163d2dd445c8792640a1d5200cf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.160959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:13.163075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 89/175] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:12 2022] GET /identity/v3/users/c86cb163d2dd445c8792640a1d5200cf => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:13.166360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.175764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb274eb-74b2-43a7-aa4e-27bae8d77f44 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.220188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.220554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.220835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.221104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] PATH_INFO: `/v3/users/81f3cbebf5174585af562fe7c97475ad` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.222631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] RBAC: Authorizing `identity:get_user(user_id=81f3cbebf5174585af562fe7c97475ad)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.223851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:13.225628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 87/176] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:13 2022] GET /identity/v3/users/81f3cbebf5174585af562fe7c97475ad => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:13.226446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.283878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.296208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e0196-2615-4bfa-b927-edfde3e773d3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.338437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.339922 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.340028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35ade6-5a76-4d77-a135-dbc5d07fb072 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.340498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.340954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.341963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:13.343281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.344872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:13.360070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 90/177] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:13.360255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.379506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.380200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.380766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.380766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.381886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.383415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:13.385147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 88/178] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:13 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:13.386010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.409957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.488374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6746-137e-4a4b-9518-93dcb15ec358 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.492933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a74d1-2b4c-4080-9a8e-e7bb732c60ef None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.528992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.529695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.529977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.530261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] PATH_INFO: `/v3/users/81f3cbebf5174585af562fe7c97475ad` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.531073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.531551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] RBAC: Authorizing `identity:get_user(user_id=81f3cbebf5174585af562fe7c97475ad)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.532146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.532630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.533109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:13.533215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.534855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 89/179] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:13 2022] GET /identity/v3/users/81f3cbebf5174585af562fe7c97475ad => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:13.535581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.559035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.616421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.618979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:13.628026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.641940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe247e-5021-44ef-af96-e28f67ba6466 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.679522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.680258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.680531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.680844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db/users/c86cb163d2dd445c8792640a1d5200cf/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.682801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6d647839334404c995bca069f96e0db, user_id=c86cb163d2dd445c8792640a1d5200cf, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.684290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:13.700599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 90/180] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:13 2022] PUT /identity/v3/projects/a6d647839334404c995bca069f96e0db/users/c86cb163d2dd445c8792640a1d5200cf/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:13.701519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.749081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 91/181] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:13.758966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:13.815528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf25924-d2ce-4cda-86c1-03231598a038 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.852748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:13.853364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:13.853642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:13.853914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:13.855121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.876320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:13.959670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65050a02-485f-4f86-a473-eafc3984ef5a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:13.972052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:13.976851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:14.034221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:14.034896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:14.035127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:14.035520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:14.036823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:14.038161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:14.040438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 92/182] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:13 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:14.045764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:14.111118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 91/183] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:14.116632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:14.177885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9581e4-d4d0-4478-afa6-7322e1827e5a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:14.217289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:14.222892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:14.246162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:14.246761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:14.247091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:14.247385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:14.248096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=a6d647839334404c995bca069f96e0db, user.id=81f3cbebf5174585af562fe7c97475ad) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:14.248395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:14.249823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:14.262862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 93/184] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:14 2022] GET /identity/v3/role_assignments?user.id=81f3cbebf5174585af562fe7c97475ad&scope.project.id=a6d647839334404c995bca069f96e0db&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:14.301559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadfe25-2c1b-4a89-90c6-18f3a265f9b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:14.340701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286abc2-0125-4177-8f84-c7eab9ad0842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:14.341344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286abc2-0125-4177-8f84-c7eab9ad0842 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:14.341618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286abc2-0125-4177-8f84-c7eab9ad0842 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:14.341889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286abc2-0125-4177-8f84-c7eab9ad0842 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:14.343073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286abc2-0125-4177-8f84-c7eab9ad0842 None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:14.344596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286abc2-0125-4177-8f84-c7eab9ad0842 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:14.346425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 92/185] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:14 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:14.352533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:14.462651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6ef08-98d9-446d-93dd-6398842cb54e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:14.500027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e216f2-3ef4-473c-95b9-f8acbc18e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:14.500634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e216f2-3ef4-473c-95b9-f8acbc18e205 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:14.500913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e216f2-3ef4-473c-95b9-f8acbc18e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:14.501183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e216f2-3ef4-473c-95b9-f8acbc18e205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:14.501897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e216f2-3ef4-473c-95b9-f8acbc18e205 None admin] RBAC: Adding query filter params (role.id=ca9ae3dc2d9b42c0b35eca1fda566496, scope.project.id=a6d647839334404c995bca069f96e0db, user.id=81f3cbebf5174585af562fe7c97475ad) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:14.502200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e216f2-3ef4-473c-95b9-f8acbc18e205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:14.503714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e216f2-3ef4-473c-95b9-f8acbc18e205 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:14.509962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 94/186] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:14 2022] GET /identity/v3/role_assignments?user.id=81f3cbebf5174585af562fe7c97475ad&scope.project.id=a6d647839334404c995bca069f96e0db&role.id=ca9ae3dc2d9b42c0b35eca1fda566496 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:15.332004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d7513-e905-4057-9bbe-c9dec1815ff2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:15.332533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d7513-e905-4057-9bbe-c9dec1815ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:15.332533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d7513-e905-4057-9bbe-c9dec1815ff2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:15.334017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 93/187] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:15.339057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8efe1-e732-4372-9b86-c3c022da528b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:15.339501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8efe1-e732-4372-9b86-c3c022da528b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:15.339694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8efe1-e732-4372-9b86-c3c022da528b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:15.353354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1e8f5-cd1f-4205-b744-148c1d5e6c66 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1e8f5-cd1f-4205-b744-148c1d5e6c66 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:15.353919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1e8f5-cd1f-4205-b744-148c1d5e6c66 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:15.355935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 94/188] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:15.358582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3fc8efe1-e732-4372-9b86-c3c022da528b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:15.359883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3fc8efe1-e732-4372-9b86-c3c022da528b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:15.360715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402044-f816-401a-99f5-054c8d25d0e1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:15.360999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402044-f816-401a-99f5-054c8d25d0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:15.361271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402044-f816-401a-99f5-054c8d25d0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:15.379158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d7402044-f816-401a-99f5-054c8d25d0e1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:15.380359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d7402044-f816-401a-99f5-054c8d25d0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:15.411283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:15.441771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 95/189] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:15.450730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 95/190] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:15.718478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8efe1-e732-4372-9b86-c3c022da528b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:15.720458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402044-f816-401a-99f5-054c8d25d0e1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:15.772436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:15.830672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402044-f816-401a-99f5-054c8d25d0e1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:15.830867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8efe1-e732-4372-9b86-c3c022da528b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:15.866018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:15.866751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:15.867035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:15.867425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:15.868812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:15.869434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:15.869711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:15.870748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:15.871637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:15.873082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:15.874661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 96/191] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:15.879041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:15.880117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:15.884043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:15.884929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:15.885856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 96/192] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:15.890693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:15.936143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:15.992512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36955d7b-6d8a-401c-8edb-70fb8e504893 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:15.998292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae86c8-3359-479d-a79c-19b92086fbe4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:16.005696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:16.031082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84934a4-8986-4399-b1e5-8fc819f9aa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:16.031713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84934a4-8986-4399-b1e5-8fc819f9aa11 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.031994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84934a4-8986-4399-b1e5-8fc819f9aa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.032265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84934a4-8986-4399-b1e5-8fc819f9aa11 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.032907 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84934a4-8986-4399-b1e5-8fc819f9aa11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:16.033110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:16.033654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.033937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.034211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.034372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84934a4-8986-4399-b1e5-8fc819f9aa11 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:16.034868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:16.035169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:16.036610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:16.045554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 97/193] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:16.051023 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:16.059886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 97/194] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:15 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:16.172067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb96c8-d032-4831-8b8d-d91456223e48 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:16.204074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d098e-302a-4a3d-9bea-a1bc301fb581 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.204404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d098e-302a-4a3d-9bea-a1bc301fb581 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.204659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d098e-302a-4a3d-9bea-a1bc301fb581 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.205988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 98/195] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:16.211081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:16.211600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92dddf-9fe3-446d-802e-32e39b9c4e23 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.211786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.211901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92dddf-9fe3-446d-802e-32e39b9c4e23 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.212058 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.212151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92dddf-9fe3-446d-802e-32e39b9c4e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.212295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] PATH_INFO: `/v3/users/42580ed213af4ef0a621ba4329ae2e16` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.228314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] RBAC: Authorizing `identity:get_user(user_id=42580ed213af4ef0a621ba4329ae2e16)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:16.229846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:16.231574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 98/196] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:16 2022] GET /identity/v3/users/42580ed213af4ef0a621ba4329ae2e16 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:16.236182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9a92dddf-9fe3-446d-802e-32e39b9c4e23 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:16.236182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9a92dddf-9fe3-446d-802e-32e39b9c4e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:16.237017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:16.268258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:16.319530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 99/197] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:16.349514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380922ef-758e-47d0-b479-735ca58cece7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:16.385999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:16.386637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.386911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.387183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.388382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:16.389864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:16.391540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 99/198] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:16 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:16.397384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92dddf-9fe3-446d-802e-32e39b9c4e23 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:16.429815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:16.513029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92dddf-9fe3-446d-802e-32e39b9c4e23 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:16.549952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:16.550584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.550868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.551137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db/users/42580ed213af4ef0a621ba4329ae2e16/roles/502d111dad8c4856b928ecc094b02d26` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.553155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] RBAC: Authorizing `identity:create_grant(project_id=a6d647839334404c995bca069f96e0db, user_id=42580ed213af4ef0a621ba4329ae2e16, role_id=502d111dad8c4856b928ecc094b02d26)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:16.554648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:16.575378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 100/199] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:16 2022] PUT /identity/v3/projects/a6d647839334404c995bca069f96e0db/users/42580ed213af4ef0a621ba4329ae2e16/roles/502d111dad8c4856b928ecc094b02d26 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:16.593573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:16.702768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:16.710606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359bb0-3b6b-4ed8-98a9-9b2e9da4b877 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:16.752366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:16.753122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.753402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.753674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.764720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:16.769947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:16.770839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:16.772251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 100/200] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:16.776978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:16.874674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:16.885107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df20b0-e1d3-46ee-85bb-5b2b30a6776a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:16.922385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:16.923046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:16.923358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:16.923675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:16.924340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:16.924640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:16.926007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:16.934980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 101/201] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:16.941039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:17.050614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6200-c0c6-40d1-a1e9-94227ee429dd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:17.086353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:17.087375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:17.087896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:17.088370 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] PATH_INFO: `/v3/users/c86cb163d2dd445c8792640a1d5200cf` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:17.090220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] RBAC: Authorizing `identity:get_user(user_id=c86cb163d2dd445c8792640a1d5200cf)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:17.091915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:17.094078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 101/202] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:16 2022] GET /identity/v3/users/c86cb163d2dd445c8792640a1d5200cf => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:17.099364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:17.217072 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d041a4c-27af-4394-959d-9af3145c33d7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:17.252057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:17.252969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:17.253476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:17.253949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:17.278829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:17.324612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:17.326428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:17.427379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 102/203] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:17.436321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:17.463940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:17.544599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d21f5a-bb4c-4f77-b882-9c27c381b44c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:17.580512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba8d14-dc0b-4a22-b011-c8af4a8597b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:17.581172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba8d14-dc0b-4a22-b011-c8af4a8597b3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:17.581452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba8d14-dc0b-4a22-b011-c8af4a8597b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:17.581724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba8d14-dc0b-4a22-b011-c8af4a8597b3 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:17.582969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba8d14-dc0b-4a22-b011-c8af4a8597b3 None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:17.584498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba8d14-dc0b-4a22-b011-c8af4a8597b3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:17.586579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 102/204] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:17 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:17.592324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:17.649158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:17.695037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f045070-5c0f-480a-a8ba-ed10e378fda7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:17.728319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f5968-b09e-4678-8000-1f7f5818ac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:17.728920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f5968-b09e-4678-8000-1f7f5818ac28 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:17.729724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f5968-b09e-4678-8000-1f7f5818ac28 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:17.729995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f5968-b09e-4678-8000-1f7f5818ac28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:17.730703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f5968-b09e-4678-8000-1f7f5818ac28 None admin] RBAC: Adding query filter params (role.id=84eb2fe76ba84dfab84d2d1ab664754f, scope.project.id=a6d647839334404c995bca069f96e0db, user.id=c86cb163d2dd445c8792640a1d5200cf) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:17.731005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f5968-b09e-4678-8000-1f7f5818ac28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:17.732480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f5968-b09e-4678-8000-1f7f5818ac28 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:17.738581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 103/205] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:17 2022] GET /identity/v3/role_assignments?user.id=c86cb163d2dd445c8792640a1d5200cf&scope.project.id=a6d647839334404c995bca069f96e0db&role.id=84eb2fe76ba84dfab84d2d1ab664754f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:18.424706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b4b40-e6ff-40a3-a7d9-0cbdd2e215e3 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.425058 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b4b40-e6ff-40a3-a7d9-0cbdd2e215e3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.425228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b4b40-e6ff-40a3-a7d9-0cbdd2e215e3 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:18.426668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 103/206] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:18.431811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f11d48-e62b-4474-9c3b-abdb6544cd6d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.432298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f11d48-e62b-4474-9c3b-abdb6544cd6d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.432511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f11d48-e62b-4474-9c3b-abdb6544cd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:18.451290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-25f11d48-e62b-4474-9c3b-abdb6544cd6d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:18.452648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-25f11d48-e62b-4474-9c3b-abdb6544cd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:18.469769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314a3ef-9461-4b97-aee8-d7d3eef8376a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.470251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314a3ef-9461-4b97-aee8-d7d3eef8376a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.470339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314a3ef-9461-4b97-aee8-d7d3eef8376a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:18.471749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 104/207] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:18.476634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8910-0861-4e7d-ac36-7c2558fcf82c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.476920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8910-0861-4e7d-ac36-7c2558fcf82c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.477199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8910-0861-4e7d-ac36-7c2558fcf82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:18.495198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4e5c8910-0861-4e7d-ac36-7c2558fcf82c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:18.496420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4e5c8910-0861-4e7d-ac36-7c2558fcf82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:18.524380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 104/208] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:18.568881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 105/209] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:18.796841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f11d48-e62b-4474-9c3b-abdb6544cd6d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:18.842374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8910-0861-4e7d-ac36-7c2558fcf82c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:18.847697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:18.864964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:18.901572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f11d48-e62b-4474-9c3b-abdb6544cd6d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:18.934792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15419174-4ad8-4287-b0df-5d0669a8d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:18.935428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419174-4ad8-4287-b0df-5d0669a8d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.935701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419174-4ad8-4287-b0df-5d0669a8d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.935967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419174-4ad8-4287-b0df-5d0669a8d8a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:18.941326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8910-0861-4e7d-ac36-7c2558fcf82c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:18.945886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15419174-4ad8-4287-b0df-5d0669a8d8a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-15419174-4ad8-4287-b0df-5d0669a8d8a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:18.951313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:18.952205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:18.953082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 105/210] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:18.954134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77f29a-2571-4d63-8d68-b08d7c74fc1a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.954369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77f29a-2571-4d63-8d68-b08d7c74fc1a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.954611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77f29a-2571-4d63-8d68-b08d7c74fc1a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:18.957270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 106/211] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:18.958410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77f29a-2571-4d63-8d68-b08d7c74fc1a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:18.978444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e25f93-08c8-45bb-b401-ba23c89de4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:18.979112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e25f93-08c8-45bb-b401-ba23c89de4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.979425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e25f93-08c8-45bb-b401-ba23c89de4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.979698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e25f93-08c8-45bb-b401-ba23c89de4d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:18.980772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e25f93-08c8-45bb-b401-ba23c89de4d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:18.982221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e25f93-08c8-45bb-b401-ba23c89de4d3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:18.983772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 106/212] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:18.987398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c0a2-5b56-402c-996b-3bdc0c1646a2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:18.987398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c0a2-5b56-402c-996b-3bdc0c1646a2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:18.987398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c0a2-5b56-402c-996b-3bdc0c1646a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.002497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2555c0a2-5b56-402c-996b-3bdc0c1646a2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:19.003715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2555c0a2-5b56-402c-996b-3bdc0c1646a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:19.061707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77f29a-2571-4d63-8d68-b08d7c74fc1a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:19.068788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 107/213] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:19.069565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555c0a2-5b56-402c-996b-3bdc0c1646a2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:19.071169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:19.089623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:19.100410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:19.101011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:19.101288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:19.101556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.102204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:19.102502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:19.103903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:19.112931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 107/214] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:19.120897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:19.186078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555c0a2-5b56-402c-996b-3bdc0c1646a2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:19.220299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:19.220898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:19.221172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:19.221440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.222074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:19.223551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:19.225246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aede6-266c-447a-ad41-05564f15891e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:19.244543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 108/215] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:19 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:19.265684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:19.266336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:19.266611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:19.266873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] PATH_INFO: `/v3/users/0f909db10fe94ffba7a14a10cb787c69` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.283874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] RBAC: Authorizing `identity:get_user(user_id=0f909db10fe94ffba7a14a10cb787c69)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:19.285363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:19.287316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 108/216] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:19 2022] GET /identity/v3/users/0f909db10fe94ffba7a14a10cb787c69 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:19.293190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:19.340373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:19.372576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:19.379016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:19.408847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ea0a4-0a70-4dd1-aa40-1f5438390b94 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:19.455181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:19.455813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:19.456086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:19.456352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.457063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] RBAC: Adding query filter params (role.id=84eb2fe76ba84dfab84d2d1ab664754f, user.id=0f909db10fe94ffba7a14a10cb787c69, scope.system=all) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:19.457365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:19.458891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:19.459654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ec794-9b29-4274-b5b0-5368e5999b7b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:19.464660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 109/217] 10.222.0.36 () {56 vars in 1308 bytes} [Mon Oct 17 06:50:19 2022] GET /identity/v3/role_assignments?user.id=0f909db10fe94ffba7a14a10cb787c69&scope.system=all&role.id=84eb2fe76ba84dfab84d2d1ab664754f => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:19.499054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:19.499768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:19.500049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:19.500335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.513231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:19.518175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:19.519100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:19.519950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 109/218] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:19.527421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:19.653110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07607251-b7e8-4ceb-a985-2c24641a8fca None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:19.694492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:19.695585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:19.696041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:19.696491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.697373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:19.698414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:19.700052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:19.706267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:19.710672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 110/219] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:19.716619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:19.812963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:19.831145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5e67c-aa2d-43d2-8e1b-1e4bfc0e6381 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:19.867347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:19.867985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:19.868261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:19.868531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] PATH_INFO: `/v3/users/42580ed213af4ef0a621ba4329ae2e16` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:19.869706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] RBAC: Authorizing `identity:get_user(user_id=42580ed213af4ef0a621ba4329ae2e16)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:19.871153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:19.872763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 110/220] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:19 2022] GET /identity/v3/users/42580ed213af4ef0a621ba4329ae2e16 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:19.878437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:19.982465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa160c83-5e8f-4cce-8e3f-1e2648615b08 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:20.017632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:20.018244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:20.018563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:20.018781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:20.085906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:20.091801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:20.093326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:20.202510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 111/221] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:20.208138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:20.312157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2480b0e-bf27-475f-9ddc-dcf2b687b972 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:20.352047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79477704-3e98-4424-b2c0-ee7630f6e075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:20.352708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79477704-3e98-4424-b2c0-ee7630f6e075 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:20.352986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79477704-3e98-4424-b2c0-ee7630f6e075 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:20.353257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79477704-3e98-4424-b2c0-ee7630f6e075 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:20.354536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79477704-3e98-4424-b2c0-ee7630f6e075 None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:20.356078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79477704-3e98-4424-b2c0-ee7630f6e075 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:20.358009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 111/222] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:20 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:20.364165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:20.471069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:20.474809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e39e3-3e81-45f7-96e2-4c1ad84b712a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:20.510943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49325a-28b0-4d4b-99b2-e292c3fc6a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:20.511567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49325a-28b0-4d4b-99b2-e292c3fc6a10 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:20.511843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49325a-28b0-4d4b-99b2-e292c3fc6a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:20.512113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49325a-28b0-4d4b-99b2-e292c3fc6a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:20.512823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49325a-28b0-4d4b-99b2-e292c3fc6a10 None admin] RBAC: Adding query filter params (role.id=502d111dad8c4856b928ecc094b02d26, scope.project.id=a6d647839334404c995bca069f96e0db, user.id=42580ed213af4ef0a621ba4329ae2e16) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:20.513132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49325a-28b0-4d4b-99b2-e292c3fc6a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:20.514633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49325a-28b0-4d4b-99b2-e292c3fc6a10 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:20.520878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 112/223] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:20 2022] GET /identity/v3/role_assignments?user.id=42580ed213af4ef0a621ba4329ae2e16&scope.project.id=a6d647839334404c995bca069f96e0db&role.id=502d111dad8c4856b928ecc094b02d26 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:21.641551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9477ad-cb43-4f66-ae47-e9d77c5fcac3 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:21.642051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9477ad-cb43-4f66-ae47-e9d77c5fcac3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:21.642051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9477ad-cb43-4f66-ae47-e9d77c5fcac3 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:21.643434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 112/224] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:21.645242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d9de0-f4e8-4a32-96dd-ceebde46f4bd None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:21.645646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d9de0-f4e8-4a32-96dd-ceebde46f4bd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:21.645838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d9de0-f4e8-4a32-96dd-ceebde46f4bd None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:21.647222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 113/225] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:21.648607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06857-f131-4531-a828-bcc5d728efb9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:21.648886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06857-f131-4531-a828-bcc5d728efb9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:21.649155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06857-f131-4531-a828-bcc5d728efb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:21.653384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f9f29-3716-4715-8922-ec5041245678 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:21.653665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f9f29-3716-4715-8922-ec5041245678 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:21.653931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f9f29-3716-4715-8922-ec5041245678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:21.668766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-acf06857-f131-4531-a828-bcc5d728efb9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:21.669945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-acf06857-f131-4531-a828-bcc5d728efb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:21.673972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-171f9f29-3716-4715-8922-ec5041245678 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:21.675127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-171f9f29-3716-4715-8922-ec5041245678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:21.690808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:21.744579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 114/226] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:21.744769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 113/227] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:21.746893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853306d-0dbb-43c2-ac1c-43a941160038 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:21.747175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853306d-0dbb-43c2-ac1c-43a941160038 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:21.747474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853306d-0dbb-43c2-ac1c-43a941160038 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:21.748846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 115/228] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:21.754199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0a48-69ea-45be-aee2-216d58758cb2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:21.754563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0a48-69ea-45be-aee2-216d58758cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:21.754908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0a48-69ea-45be-aee2-216d58758cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:21.773032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8d8a0a48-69ea-45be-aee2-216d58758cb2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:21.774204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8d8a0a48-69ea-45be-aee2-216d58758cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:21.843073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 114/229] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:22.009984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853306d-0dbb-43c2-ac1c-43a941160038 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.028917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a0a48-69ea-45be-aee2-216d58758cb2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.106552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:22.114625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:22.115539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853306d-0dbb-43c2-ac1c-43a941160038 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.130089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a0a48-69ea-45be-aee2-216d58758cb2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.152185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.152813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.153086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.153350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.163748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:22.166787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.167464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.167743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.168009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:22.168646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:22.169659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:22.170443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 116/230] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:22.171100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.179941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:22.185737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 115/231] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:22.186526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.268095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5250-f31c-469b-abfa-5fe69e8a62ab None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.288146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518c424-0d96-4a6b-93c5-1deb42363448 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.300743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.301358 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.301629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.301896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.302979 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.304466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.306162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 117/232] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:22.306741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.324715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.325318 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.325592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.325864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.326512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:22.326812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.328228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.331779 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:22.341470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 116/233] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:22.341470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.349856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:22.406687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67d8c-8edd-4f88-9602-ba41b7431539 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.437374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3fc14-c666-4972-b711-e42b9649cd39 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.439636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.440239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.440519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.440790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.441437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:22.441736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.443103 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.451306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 118/234] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:22.452078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.470310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.470905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.471184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.471482 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] PATH_INFO: `/v3/groups` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.472098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.473517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.489317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 117/235] 10.222.0.36 () {60 vars in 1134 bytes} [Mon Oct 17 06:50:22 2022] POST /identity/v3/groups => generated 229 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:22.490101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.540203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:22.558863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55be2c-8703-4ac8-84b8-169ad39e43e7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.567745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:22.596099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.596744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.597020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.597291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] PATH_INFO: `/v3/users/0f909db10fe94ffba7a14a10cb787c69` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.598444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] RBAC: Authorizing `identity:get_user(user_id=0f909db10fe94ffba7a14a10cb787c69)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.599954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.601574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 119/236] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/users/0f909db10fe94ffba7a14a10cb787c69 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:22.607647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.612085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae40fc-8e4e-4468-ae69-6fdbcbc27c15 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.650362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.651617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.651874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.652055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] PATH_INFO: `/v3/users/31cc0d7464ba430089dc586baa4dd83a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.669412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] RBAC: Authorizing `identity:get_user(user_id=31cc0d7464ba430089dc586baa4dd83a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.670921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.672090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 118/237] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/users/31cc0d7464ba430089dc586baa4dd83a => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:22.678273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:22.717830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7e78-07f2-4db4-afd4-e6eb3a266cef None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.752330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6668302-9583-4fa9-8532-23528b1d7414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.752983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6668302-9583-4fa9-8532-23528b1d7414 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.753268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6668302-9583-4fa9-8532-23528b1d7414 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.753594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6668302-9583-4fa9-8532-23528b1d7414 None admin] PATH_INFO: `/v3/system/users/0f909db10fe94ffba7a14a10cb787c69/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.755008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6668302-9583-4fa9-8532-23528b1d7414 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f909db10fe94ffba7a14a10cb787c69, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.756428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6668302-9583-4fa9-8532-23528b1d7414 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.767438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 120/238] 10.222.0.36 () {58 vars in 1334 bytes} [Mon Oct 17 06:50:22 2022] PUT /identity/v3/system/users/0f909db10fe94ffba7a14a10cb787c69/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:50:22.793225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa797bf-28a3-43a7-b88f-74ae7084882b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:22.821858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:22.836005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26aa5a-1fad-413c-a27f-6a6326fe92bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:22.836670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26aa5a-1fad-413c-a27f-6a6326fe92bd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:22.836946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26aa5a-1fad-413c-a27f-6a6326fe92bd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:22.837210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26aa5a-1fad-413c-a27f-6a6326fe92bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:22.837935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26aa5a-1fad-413c-a27f-6a6326fe92bd None admin] RBAC: Adding query filter params (role.id=502d111dad8c4856b928ecc094b02d26, user.id=31cc0d7464ba430089dc586baa4dd83a, scope.system=all) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:22.838392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26aa5a-1fad-413c-a27f-6a6326fe92bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:22.839872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26aa5a-1fad-413c-a27f-6a6326fe92bd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:22.846065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 119/239] 10.222.0.36 () {56 vars in 1308 bytes} [Mon Oct 17 06:50:22 2022] GET /identity/v3/role_assignments?user.id=31cc0d7464ba430089dc586baa4dd83a&scope.system=all&role.id=502d111dad8c4856b928ecc094b02d26 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:24.668133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f432e-11a7-4866-8f90-7f2040d71b41 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:24.668627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f432e-11a7-4866-8f90-7f2040d71b41 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:24.668721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f432e-11a7-4866-8f90-7f2040d71b41 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:24.670144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 121/240] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:24.675079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021b524-36b1-42e1-9031-f45262a2ce88 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:24.675508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021b524-36b1-42e1-9031-f45262a2ce88 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:24.675678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021b524-36b1-42e1-9031-f45262a2ce88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:24.694667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a021b524-36b1-42e1-9031-f45262a2ce88 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:24.695858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a021b524-36b1-42e1-9031-f45262a2ce88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:24.763015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 120/241] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:24.894450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1f81d-a49a-452e-89c0-fd0cda31567b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:24.894807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1f81d-a49a-452e-89c0-fd0cda31567b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:24.894985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1f81d-a49a-452e-89c0-fd0cda31567b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:24.896508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 122/242] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:24.902451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c238aa-d26c-4fa2-8686-43f75d556d46 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:24.902725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c238aa-d26c-4fa2-8686-43f75d556d46 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:24.902994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c238aa-d26c-4fa2-8686-43f75d556d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:24.920805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-35c238aa-d26c-4fa2-8686-43f75d556d46 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:24.921890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-35c238aa-d26c-4fa2-8686-43f75d556d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:24.984146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacde4c-6438-4b60-a428-3fffc30908e0 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:24.984463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacde4c-6438-4b60-a428-3fffc30908e0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:24.984680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacde4c-6438-4b60-a428-3fffc30908e0 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:24.986182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 123/243] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:24.987500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 121/244] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:24.995393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e6157-9be3-4491-97af-89127412a198 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:24.995750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e6157-9be3-4491-97af-89127412a198 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:24.996107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e6157-9be3-4491-97af-89127412a198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.013797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-734e6157-9be3-4491-97af-89127412a198 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:25.015046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-734e6157-9be3-4491-97af-89127412a198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:25.025926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.030903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c238aa-d26c-4fa2-8686-43f75d556d46 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.088382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 124/245] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:25.089522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.142297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c238aa-d26c-4fa2-8686-43f75d556d46 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.180469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.181040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.181315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.181582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.182718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:25.184262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:25.187250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 122/246] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:25.191624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e6157-9be3-4491-97af-89127412a198 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.259493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.293652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e6157-9be3-4491-97af-89127412a198 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.329053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.329644 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.329917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.330186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] PATH_INFO: `/v3/groups` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.330807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:25.332304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:25.350567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 125/247] 10.222.0.36 () {60 vars in 1134 bytes} [Mon Oct 17 06:50:25 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:25.360144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.367923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db4ba9-d0d4-4a3c-acbf-7321d5e1836e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.374819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.389575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.410504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.411154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.411461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.411730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.423518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:25.428450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:25.429429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:25.430453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 123/248] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:25.436223 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.490021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdeaa9-c604-4462-bc88-6981cabdfb9a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.528599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.529494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.529767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.530029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.542900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:25.547476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8292d-c42f-45d2-9fb2-6bad474c7914 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:25.548199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:25.549084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:25.549878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 126/249] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:25.555120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.589028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.589637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.589915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.590183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.590836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:25.591403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:25.592527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:25.602200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 124/250] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:25.609388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.654034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.659555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.688918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91561afd-630d-4141-82ab-80cbd4f40851 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.727468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41df2b0-d8c2-4fd4-912b-e9c65e068e05 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.728310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.729441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.729880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.730150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.731675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:25.731980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:25.733390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:25.744442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 127/251] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:25.752196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.770140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.770778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.771055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.773593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] PATH_INFO: `/v3/users/0f909db10fe94ffba7a14a10cb787c69` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.774266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] RBAC: Authorizing `identity:get_user(user_id=0f909db10fe94ffba7a14a10cb787c69)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:25.776436 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:25.779008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 125/252] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/users/0f909db10fe94ffba7a14a10cb787c69 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:25.784975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.877666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b2b6d-97c6-4259-85b1-7855c69e6151 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.914754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.927238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.927872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdcaee-020e-40d5-83ab-fb7219cdc02c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:25.927981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.928184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.928434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] PATH_INFO: `/v3/users/31cc0d7464ba430089dc586baa4dd83a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.929570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] RBAC: Authorizing `identity:get_user(user_id=31cc0d7464ba430089dc586baa4dd83a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:25.931027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:25.932936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 128/253] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/users/31cc0d7464ba430089dc586baa4dd83a => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:25.938365 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:25.944390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:25.970514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9a22d-e60e-42b2-8e57-237e789ef575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:25.971125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9a22d-e60e-42b2-8e57-237e789ef575 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:25.971427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9a22d-e60e-42b2-8e57-237e789ef575 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:25.971696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9a22d-e60e-42b2-8e57-237e789ef575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:25.972411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9a22d-e60e-42b2-8e57-237e789ef575 None admin] RBAC: Adding query filter params (role.id=84eb2fe76ba84dfab84d2d1ab664754f, user.id=0f909db10fe94ffba7a14a10cb787c69, scope.system=all) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:25.972774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9a22d-e60e-42b2-8e57-237e789ef575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:25.974205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9a22d-e60e-42b2-8e57-237e789ef575 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:25.987695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 126/254] 10.222.0.36 () {56 vars in 1308 bytes} [Mon Oct 17 06:50:25 2022] GET /identity/v3/role_assignments?user.id=0f909db10fe94ffba7a14a10cb787c69&scope.system=all&role.id=84eb2fe76ba84dfab84d2d1ab664754f => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:26.096452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef42c7-6b69-40ad-84fe-cd93e3163c84 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:26.140406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd606a-0d77-4a72-833b-6f09752927b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:26.141487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd606a-0d77-4a72-833b-6f09752927b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:26.141961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd606a-0d77-4a72-833b-6f09752927b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:26.142424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd606a-0d77-4a72-833b-6f09752927b5 None admin] PATH_INFO: `/v3/system/users/31cc0d7464ba430089dc586baa4dd83a/roles/502d111dad8c4856b928ecc094b02d26` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:26.145200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd606a-0d77-4a72-833b-6f09752927b5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=31cc0d7464ba430089dc586baa4dd83a, role_id=502d111dad8c4856b928ecc094b02d26)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:26.146922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd606a-0d77-4a72-833b-6f09752927b5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:26.165373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 129/255] 10.222.0.36 () {58 vars in 1334 bytes} [Mon Oct 17 06:50:25 2022] PUT /identity/v3/system/users/31cc0d7464ba430089dc586baa4dd83a/roles/502d111dad8c4856b928ecc094b02d26 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:27.737361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d60da-2ccd-4c51-8800-cf2d466e8fc2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:27.737865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d60da-2ccd-4c51-8800-cf2d466e8fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:27.737939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d60da-2ccd-4c51-8800-cf2d466e8fc2 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:27.739619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 127/256] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:27.745185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5f7c-919d-462e-8060-1f325f0524fa None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:27.745974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5f7c-919d-462e-8060-1f325f0524fa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:27.746448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5f7c-919d-462e-8060-1f325f0524fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:27.776772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c4eb5f7c-919d-462e-8060-1f325f0524fa None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:27.778331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4eb5f7c-919d-462e-8060-1f325f0524fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:27.853912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 130/257] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:50:28.163385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d60da-2ccd-4c51-8800-cf2d466e8fc2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.181040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d60da-2ccd-4c51-8800-cf2d466e8fc2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.186291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.186910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.187189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.187730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.202776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:28.208127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:28.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:28.209768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 128/258] 10.222.0.36 () {56 vars in 1120 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:28.216721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb5f7c-919d-462e-8060-1f325f0524fa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.231232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb5f7c-919d-462e-8060-1f325f0524fa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.241158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.242033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.242510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.242987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.243880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:28.244959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:28.246930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:50:28.246930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 06:50:28.246930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 06:50:28.248215 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:28.250022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:28.264845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 131/259] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:28.270919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.284397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3cd52-c4c1-4c18-bfcf-f851be3a2752 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.294824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d386d-234e-4a04-a419-d54a4d0f6551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.295528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d386d-234e-4a04-a419-d54a4d0f6551 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.295812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d386d-234e-4a04-a419-d54a4d0f6551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.296081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d386d-234e-4a04-a419-d54a4d0f6551 admin admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.305871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d386d-234e-4a04-a419-d54a4d0f6551 admin admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:28.307786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-354d386d-234e-4a04-a419-d54a4d0f6551 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:50:28.307786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 06:50:28.307786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 06:50:28.309034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d386d-234e-4a04-a419-d54a4d0f6551 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:28.311112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 129/260] 10.222.0.36 () {56 vars in 1201 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:28.316944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.325705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36353d-c34a-4cd2-b110-b04e39f527e5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.325998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36353d-c34a-4cd2-b110-b04e39f527e5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.326278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36353d-c34a-4cd2-b110-b04e39f527e5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.327798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 130/261] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:28.328939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9d88-ed4f-4df3-8612-4d627ff495ce admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.335472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf054-02fc-4734-a97d-d03c0f30b7b3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.336245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf054-02fc-4734-a97d-d03c0f30b7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.336524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf054-02fc-4734-a97d-d03c0f30b7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.344659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.345308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.345588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.345932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.348085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:28.350200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:28.352050 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 132/262] 10.222.0.36 () {56 vars in 1207 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:28.358784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.371552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-199cf054-02fc-4734-a97d-d03c0f30b7b3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:28.373389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-199cf054-02fc-4734-a97d-d03c0f30b7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:28.382519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44041d-faf4-4182-97aa-2ad91fd8bc6d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.389832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 131/263] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:50:28.396875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78b090-ab98-424b-b0d1-e6ba398b144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.397834 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b090-ab98-424b-b0d1-e6ba398b144e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.398310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b090-ab98-424b-b0d1-e6ba398b144e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.398780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b090-ab98-424b-b0d1-e6ba398b144e admin admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323/groups/7f2ad812715647ff95a69ebab232b279/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.401481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78b090-ab98-424b-b0d1-e6ba398b144e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ff1cdc507747e5a54adbea719b6323, group_id=7f2ad812715647ff95a69ebab232b279, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:28.402714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-0c78b090-ab98-424b-b0d1-e6ba398b144e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:50:28.402714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 06:50:28.402714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 06:50:28.405159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78b090-ab98-424b-b0d1-e6ba398b144e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:28.423902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 133/264] 10.222.0.36 () {58 vars in 1463 bytes} [Mon Oct 17 06:50:28 2022] PUT /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323/groups/7f2ad812715647ff95a69ebab232b279/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:28.691617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cf054-02fc-4734-a97d-d03c0f30b7b3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.701967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cf054-02fc-4734-a97d-d03c0f30b7b3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.703748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9961-49c0-4817-9fb1-a187c1a7224d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.704860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9961-49c0-4817-9fb1-a187c1a7224d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.705696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9961-49c0-4817-9fb1-a187c1a7224d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.711874 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 134/265] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:28.718257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878dcb3-87d4-4154-b1d4-e117151c84de None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.718813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878dcb3-87d4-4154-b1d4-e117151c84de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.719306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878dcb3-87d4-4154-b1d4-e117151c84de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.763733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0878dcb3-87d4-4154-b1d4-e117151c84de None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:28.772895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0878dcb3-87d4-4154-b1d4-e117151c84de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:28.821273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.826803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.827358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.827835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.841269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:28.847139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:28.848059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:28.848842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:28.853848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 132/266] 10.222.0.36 () {56 vars in 1135 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:28.867010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.878360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6629e-9e3b-44d4-84dd-d093359635d8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.883175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.884052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.884526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.885528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.886377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:28.886873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:28.888159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:50:28.888159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 06:50:28.888159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 06:50:28.889475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:28.894153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 135/267] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:50:28.901523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 133/268] 10.222.0.36 () {56 vars in 1132 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:28.905481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878dcb3-87d4-4154-b1d4-e117151c84de None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.921133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878dcb3-87d4-4154-b1d4-e117151c84de None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.931343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.932056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.932331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.932601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.939986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:28.941038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:50:28.941038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 06:50:28.941038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 06:50:28.942320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:28.944093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 136/269] 10.222.0.36 () {56 vars in 1201 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:28.950058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.961422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0bbcc-d3b7-4b0e-8966-92f4d32e0733 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:28.973294 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c26f8-37d8-433a-affa-14f9cf5c3702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:28.974378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c26f8-37d8-433a-affa-14f9cf5c3702 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:28.974876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c26f8-37d8-433a-affa-14f9cf5c3702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:28.975517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c26f8-37d8-433a-affa-14f9cf5c3702 admin admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:28.977098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c26f8-37d8-433a-affa-14f9cf5c3702 admin admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:28.980086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c26f8-37d8-433a-affa-14f9cf5c3702 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:28.982084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 134/270] 10.222.0.36 () {56 vars in 1207 bytes} [Mon Oct 17 06:50:28 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:28.987842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:28.997833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff0c2f-cc66-41f5-a6ec-d1d4c196531e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:29.007393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eba17d-9626-4dbc-8760-f2b14b73d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:29.008057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba17d-9626-4dbc-8760-f2b14b73d092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.008334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba17d-9626-4dbc-8760-f2b14b73d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.008602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba17d-9626-4dbc-8760-f2b14b73d092 admin admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323/groups/7f2ad812715647ff95a69ebab232b279/roles/ca9ae3dc2d9b42c0b35eca1fda566496` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.016976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eba17d-9626-4dbc-8760-f2b14b73d092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ff1cdc507747e5a54adbea719b6323, group_id=7f2ad812715647ff95a69ebab232b279, role_id=ca9ae3dc2d9b42c0b35eca1fda566496)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:29.019719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eba17d-9626-4dbc-8760-f2b14b73d092 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:29.045933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 137/271] 10.222.0.36 () {58 vars in 1463 bytes} [Mon Oct 17 06:50:28 2022] PUT /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323/groups/7f2ad812715647ff95a69ebab232b279/roles/ca9ae3dc2d9b42c0b35eca1fda566496 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:29.328416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba7a6-919e-4390-a9af-7b835ce55b9c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.329062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba7a6-919e-4390-a9af-7b835ce55b9c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.329543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba7a6-919e-4390-a9af-7b835ce55b9c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.331176 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 135/272] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:29.337292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255f50-9c84-4236-9b83-d377b6336387 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.337576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255f50-9c84-4236-9b83-d377b6336387 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.338389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255f50-9c84-4236-9b83-d377b6336387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.364245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e3255f50-9c84-4236-9b83-d377b6336387 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:29.365443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e3255f50-9c84-4236-9b83-d377b6336387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:29.446462 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 138/273] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:29.460354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba7a6-919e-4390-a9af-7b835ce55b9c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:29.552932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba7a6-919e-4390-a9af-7b835ce55b9c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:29.559341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:29.559973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.560241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.560500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.571293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:29.576646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:29.577573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:29.578377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 136/274] 10.222.0.36 () {56 vars in 1120 bytes} [Mon Oct 17 06:50:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 118 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:29.583566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255f50-9c84-4236-9b83-d377b6336387 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:29.594233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255f50-9c84-4236-9b83-d377b6336387 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:29.598525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:29.599131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.599448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.599712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.600367 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:29.600663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:29.602274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:29.614021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 139/275] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:50:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:29.619560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:29.627532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:29.631750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502b2bb-16e1-44aa-86c2-ca3780a85767 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:29.641180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:29.641806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.642082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.642348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.643494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:29.645433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:29.646990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 137/276] 10.222.0.36 () {56 vars in 1201 bytes} [Mon Oct 17 06:50:29 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:29.652550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:29.668878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f1a02-0f3e-4007-aa9b-85fa43c803d7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:29.676707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:29.677361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.677604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.677875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.678976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:29.681115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:29.682969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 140/277] 10.222.0.36 () {56 vars in 1207 bytes} [Mon Oct 17 06:50:29 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:29.688747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:29.701139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297579c-c6d7-44d3-892e-8d82e8feff96 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:29.705531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:29.706181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:29.706454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:29.706723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db/groups/7f2ad812715647ff95a69ebab232b279/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:29.708615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d647839334404c995bca069f96e0db, group_id=7f2ad812715647ff95a69ebab232b279, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:29.709624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:50:29.709624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 06:50:29.709624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 06:50:29.711633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:29.732617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 138/278] 10.222.0.36 () {58 vars in 1463 bytes} [Mon Oct 17 06:50:29 2022] PUT /identity/v3/projects/a6d647839334404c995bca069f96e0db/groups/7f2ad812715647ff95a69ebab232b279/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:29.993636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:30.030702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:30.130637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0fbc4-25aa-4ab2-8a92-4e8df0087042 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:30.188800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:30.189431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.189703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.189971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.205583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:30.211138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:30.212061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:30.212971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 141/279] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:30.219808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:30.329772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a791-5da3-4a82-81f5-efefbe55ef9e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:30.372177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8700c-7a65-4a60-9d7d-40c355114691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:30.372785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8700c-7a65-4a60-9d7d-40c355114691 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.373057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8700c-7a65-4a60-9d7d-40c355114691 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.373319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8700c-7a65-4a60-9d7d-40c355114691 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.374042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8700c-7a65-4a60-9d7d-40c355114691 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:30.374351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8700c-7a65-4a60-9d7d-40c355114691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:30.375794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8700c-7a65-4a60-9d7d-40c355114691 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:30.385325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 139/280] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:30.391922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:30.460370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4699c-0d46-4006-a5ef-c7b5a657d33f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.460707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4699c-0d46-4006-a5ef-c7b5a657d33f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.460906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4699c-0d46-4006-a5ef-c7b5a657d33f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.462264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 140/281] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:50:30.477256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f62e-b1b4-4e92-946b-3fc52034db3c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.477570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f62e-b1b4-4e92-946b-3fc52034db3c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.477824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f62e-b1b4-4e92-946b-3fc52034db3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.519720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d866f62e-b1b4-4e92-946b-3fc52034db3c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:30.520938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d866f62e-b1b4-4e92-946b-3fc52034db3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:30.552742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:30.590884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76d18-0840-4d7e-92bf-0f583605642d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:30.617093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:30.640408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 141/282] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 06:50:30.641671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abc4c-cfa0-4cee-9bfb-a5be52d6c23d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.641953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abc4c-cfa0-4cee-9bfb-a5be52d6c23d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.642222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abc4c-cfa0-4cee-9bfb-a5be52d6c23d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.643496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 142/283] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:30.651142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66641760-6eba-4a25-bd4f-de2eca56df92 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.651449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66641760-6eba-4a25-bd4f-de2eca56df92 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.651721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66641760-6eba-4a25-bd4f-de2eca56df92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.665898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:30.666571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.666850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.667120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] PATH_INFO: `/v3/users/31cc0d7464ba430089dc586baa4dd83a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.668985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] RBAC: Authorizing `identity:get_user(user_id=31cc0d7464ba430089dc586baa4dd83a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:30.670433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:30.677206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 142/284] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:50:30 2022] GET /identity/v3/users/31cc0d7464ba430089dc586baa4dd83a => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:30.678047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:30.691975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-66641760-6eba-4a25-bd4f-de2eca56df92 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:30.693216 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66641760-6eba-4a25-bd4f-de2eca56df92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:30.712412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 143/285] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:50:30.836711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ff47b-f065-450f-9570-7dee8ad57710 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:30.876002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:30.876523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:30.876799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:30.877062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:30.877780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] RBAC: Adding query filter params (role.id=502d111dad8c4856b928ecc094b02d26, user.id=31cc0d7464ba430089dc586baa4dd83a, scope.system=all) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:30.878082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:30.879544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:30.893848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 143/286] 10.222.0.36 () {56 vars in 1308 bytes} [Mon Oct 17 06:50:30 2022] GET /identity/v3/role_assignments?user.id=31cc0d7464ba430089dc586baa4dd83a&scope.system=all&role.id=502d111dad8c4856b928ecc094b02d26 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:31.022019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66641760-6eba-4a25-bd4f-de2eca56df92 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.034840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66641760-6eba-4a25-bd4f-de2eca56df92 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.039515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.040168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.040449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.040721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.058106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:31.063332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:31.064279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:31.065069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 144/287] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:31.070782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.088986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525257e-2f85-4e4f-844e-e9283bc0e326 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.099672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.100640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.101117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.101583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.102466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:31.102966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.105391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.120174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 144/288] 10.222.0.36 () {56 vars in 1120 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:31.125438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.140664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f46214-b842-45ee-b2be-dab44714ebad admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.151756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.152408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.152687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.152958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] PATH_INFO: `/v3/groups/7246ee5a489a4a2c99e935c723e436a6` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.164626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] RBAC: Authorizing `identity:get_group(group_id=7246ee5a489a4a2c99e935c723e436a6)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.167107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.168861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 145/289] 10.222.0.36 () {56 vars in 1201 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/groups/7246ee5a489a4a2c99e935c723e436a6 => generated 229 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:31.174431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.181203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:31.198314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8794e18-575a-4cf4-b929-d3507cd3e1e6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.203754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.207772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.208768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.209261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.209747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] PATH_INFO: `/v3/projects/86c1b532428347929d42c84633169ba2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.211999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] RBAC: Authorizing `identity:get_project(project_id=86c1b532428347929d42c84633169ba2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.214148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.216314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 145/290] 10.222.0.36 () {56 vars in 1207 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/projects/86c1b532428347929d42c84633169ba2 => generated 344 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:31.222451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.225095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9da39-9244-4cc2-8cbf-2fc057a5ab09 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.236853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.237856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.238353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.238819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.249403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f6ed3-0555-47fa-bf31-185c8fa68a3b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.263408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.264526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.264812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:31.265244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.265709 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] PATH_INFO: `/v3/projects/86c1b532428347929d42c84633169ba2/groups/7246ee5a489a4a2c99e935c723e436a6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:31.269933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:31.270823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:31.271646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 146/291] 10.222.0.36 () {56 vars in 1135 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:31.274045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c1b532428347929d42c84633169ba2, group_id=7246ee5a489a4a2c99e935c723e436a6, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.276871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.278155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.293018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66bd34-767b-412a-92bc-447eceea2ffa admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.301618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 146/292] 10.222.0.36 () {58 vars in 1463 bytes} [Mon Oct 17 06:50:31 2022] PUT /identity/v3/projects/86c1b532428347929d42c84633169ba2/groups/7246ee5a489a4a2c99e935c723e436a6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:31.377737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.378871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.379423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.379891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.380815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:31.381330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.383188 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.393696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 147/293] 10.222.0.36 () {56 vars in 1132 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:31.399053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.410696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451a65b-8c16-4615-ba6f-4d023a5e2a6f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.415958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.416970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.417495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.417967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.419402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.421698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.423781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 147/294] 10.222.0.36 () {56 vars in 1201 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:31.429547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.441664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b383c-81cc-4c41-8269-70196be7122f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.450419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3016489-95b4-406f-ae0c-92f1f95a856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.451532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3016489-95b4-406f-ae0c-92f1f95a856b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.452026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3016489-95b4-406f-ae0c-92f1f95a856b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.452522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3016489-95b4-406f-ae0c-92f1f95a856b admin admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.453959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3016489-95b4-406f-ae0c-92f1f95a856b admin admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.456543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3016489-95b4-406f-ae0c-92f1f95a856b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.458669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 148/295] 10.222.0.36 () {56 vars in 1207 bytes} [Mon Oct 17 06:50:31 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:31.465331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:31.476534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d65ca-59a2-4d96-bebc-03c3b2df079f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:31.482805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a3f7b-ac53-47bb-bd5a-7d780fbaeed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:31.483972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3f7b-ac53-47bb-bd5a-7d780fbaeed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:31.484515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3f7b-ac53-47bb-bd5a-7d780fbaeed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:31.485040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3f7b-ac53-47bb-bd5a-7d780fbaeed7 admin admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db/groups/7f2ad812715647ff95a69ebab232b279/roles/ca9ae3dc2d9b42c0b35eca1fda566496` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:31.487595 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a3f7b-ac53-47bb-bd5a-7d780fbaeed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d647839334404c995bca069f96e0db, group_id=7f2ad812715647ff95a69ebab232b279, role_id=ca9ae3dc2d9b42c0b35eca1fda566496)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:31.490643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a3f7b-ac53-47bb-bd5a-7d780fbaeed7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:31.514712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 148/296] 10.222.0.36 () {58 vars in 1463 bytes} [Mon Oct 17 06:50:31 2022] PUT /identity/v3/projects/a6d647839334404c995bca069f96e0db/groups/7f2ad812715647ff95a69ebab232b279/roles/ca9ae3dc2d9b42c0b35eca1fda566496 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:32.307404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f6af8-34fd-47f7-b6d8-cc3846687ea2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:32.308412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f6af8-34fd-47f7-b6d8-cc3846687ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:32.308890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f6af8-34fd-47f7-b6d8-cc3846687ea2 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:32.310518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 149/297] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:32.316307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127929-2cbd-41d9-95d2-65548fd6ec32 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:32.317115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127929-2cbd-41d9-95d2-65548fd6ec32 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:32.317642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127929-2cbd-41d9-95d2-65548fd6ec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:32.336338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-34127929-2cbd-41d9-95d2-65548fd6ec32 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:32.339145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b2c1-f077-4a00-ab53-71ee464bd599 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:32.339610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b2c1-f077-4a00-ab53-71ee464bd599 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:32.340033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b2c1-f077-4a00-ab53-71ee464bd599 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:32.341510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 150/298] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:32.342429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-34127929-2cbd-41d9-95d2-65548fd6ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:32.348567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e728379-c1f4-4a6c-85cc-e1605228405b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:32.349122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e728379-c1f4-4a6c-85cc-e1605228405b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:32.349590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e728379-c1f4-4a6c-85cc-e1605228405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:32.372993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9e728379-c1f4-4a6c-85cc-e1605228405b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:32.374600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9e728379-c1f4-4a6c-85cc-e1605228405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:32.442198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 149/299] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:32.471456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 151/300] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:32.797749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798f8a-6dfd-4ce5-bb49-67e0f68cac12 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:32.798402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798f8a-6dfd-4ce5-bb49-67e0f68cac12 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:32.798873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798f8a-6dfd-4ce5-bb49-67e0f68cac12 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:32.800674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 150/301] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:32.806572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb848a-63fb-439d-a324-a299da057a45 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:32.807076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb848a-63fb-439d-a324-a299da057a45 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:32.807716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb848a-63fb-439d-a324-a299da057a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:32.836161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-70cb848a-63fb-439d-a324-a299da057a45 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:32.838232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-70cb848a-63fb-439d-a324-a299da057a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:32.863789 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:32.914367 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed798f8a-6dfd-4ce5-bb49-67e0f68cac12 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:32.927090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 152/302] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:32.988353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb848a-63fb-439d-a324-a299da057a45 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:33.074726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:33.082285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed798f8a-6dfd-4ce5-bb49-67e0f68cac12 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.165334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.165334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.165570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.166211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.173693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb848a-63fb-439d-a324-a299da057a45 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.199826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:33.204668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:33.205546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:33.206330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 151/303] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:33.211994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:33.218539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.219461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.219943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.220406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.232924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:33.239435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:33.240297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:33.241084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:33.243416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 153/304] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:50:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:33.251685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:33.299181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:33.377948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bbdfc-cae4-4fbd-ab11-52c381d7bc7e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.407024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a8ba9-cc2a-4ea7-9d5d-9410ca88e8e9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.416983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.417938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.418436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.419055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.419955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:33.420464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:33.422032 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:33.431982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 152/305] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:33.433053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:33.449111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.449726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.449998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.450272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.450932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:33.451278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:33.452678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:33.461634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:33.462044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 154/306] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:33.463041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:33.548672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777577d-766a-4aa5-a757-fd68a530df78 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.549307 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:33.573670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828b1e2-1cb5-4f7f-9160-435e1ee4fdac None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.594764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.595806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.596285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.596752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.612091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:33.612984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.613592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.613873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.614146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:33.620519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:33.621428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:50:33.622210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:33.625121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 153/307] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:33.626612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:33.694646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:33.696470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:33.736455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:33.767171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a34672-4eae-4a02-8a83-efd422f33a00 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.790391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:33.817340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.817850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.818128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.818399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.825949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 155/308] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:33.827193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:33.909876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:33.911400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:33.951130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6625d-868d-46d5-a2fa-1495ffa1c1e5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:33.992837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:33.994829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:33.995418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:33.995841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:33.996920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:33.997390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:33.999865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.014359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 156/309] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:50:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.016547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.033925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.057625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 154/310] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:34.058560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.070119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.140738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890de2e-8dcd-4e8c-ac12-cc3d5208ed46 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.180604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49351e-6a80-4134-bc5f-159559ee0393 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.185493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.186152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.186435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.186712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.188173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.190400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.192742 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 157/311] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.193588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.223338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2af6a5-4a6a-4f25-93be-5380e1f94ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.223338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af6a5-4a6a-4f25-93be-5380e1f94ddb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.223338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af6a5-4a6a-4f25-93be-5380e1f94ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.223652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af6a5-4a6a-4f25-93be-5380e1f94ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.293627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.313155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2af6a5-4a6a-4f25-93be-5380e1f94ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.314615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2af6a5-4a6a-4f25-93be-5380e1f94ddb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.325122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84be1-3623-4c02-9c91-ad4dc201f0af None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.336791 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.365160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482041ba-f99f-4cf0-a2f5-76081fd6cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.365794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482041ba-f99f-4cf0-a2f5-76081fd6cd74 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.366073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482041ba-f99f-4cf0-a2f5-76081fd6cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.366345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482041ba-f99f-4cf0-a2f5-76081fd6cd74 None admin] PATH_INFO: `/v3/projects/66ff1cdc507747e5a54adbea719b6323` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.367480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482041ba-f99f-4cf0-a2f5-76081fd6cd74 None admin] RBAC: Authorizing `identity:get_project(project_id=66ff1cdc507747e5a54adbea719b6323)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.368939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482041ba-f99f-4cf0-a2f5-76081fd6cd74 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.370721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 158/312] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/projects/66ff1cdc507747e5a54adbea719b6323 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.371707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19997b45-e239-4d20-8814-8273cc2ab859 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.371982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19997b45-e239-4d20-8814-8273cc2ab859 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.372251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19997b45-e239-4d20-8814-8273cc2ab859 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.373450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 159/313] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:34.374142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19997b45-e239-4d20-8814-8273cc2ab859 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.433778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 155/314] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:34.437630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ba627-cafd-477b-b5f5-3a8b4a266f2b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.437630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ba627-cafd-477b-b5f5-3a8b4a266f2b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.437630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ba627-cafd-477b-b5f5-3a8b4a266f2b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.437630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 156/315] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:34.437891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ba627-cafd-477b-b5f5-3a8b4a266f2b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.489760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19997b45-e239-4d20-8814-8273cc2ab859 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.530095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69381fd-3a4a-4156-b45d-64de9f5ebe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.531193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69381fd-3a4a-4156-b45d-64de9f5ebe15 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.531605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69381fd-3a4a-4156-b45d-64de9f5ebe15 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.531881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69381fd-3a4a-4156-b45d-64de9f5ebe15 None admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.532438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.534026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69381fd-3a4a-4156-b45d-64de9f5ebe15 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.536053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69381fd-3a4a-4156-b45d-64de9f5ebe15 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.538033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 160/316] 10.222.0.36 () {56 vars in 1180 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.539304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fcef6-84a1-4615-a950-13e8e3d6100c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.539582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fcef6-84a1-4615-a950-13e8e3d6100c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.539846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fcef6-84a1-4615-a950-13e8e3d6100c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.554870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ba627-cafd-477b-b5f5-3a8b4a266f2b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.559915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-123fcef6-84a1-4615-a950-13e8e3d6100c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:34.563114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-123fcef6-84a1-4615-a950-13e8e3d6100c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:34.599611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.600298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.600579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.600860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.601964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.604017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.605900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 157/317] 10.222.0.36 () {56 vars in 1180 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.606671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.609219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.642047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 161/318] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:34.643650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f9541-7480-4dfb-bb89-5fbcd8898ead None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.643924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f9541-7480-4dfb-bb89-5fbcd8898ead None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.644189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f9541-7480-4dfb-bb89-5fbcd8898ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.671342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5a5f9541-7480-4dfb-bb89-5fbcd8898ead None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:34.672670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a5f9541-7480-4dfb-bb89-5fbcd8898ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:34.722204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd9ec0-bc0a-4d00-9566-2211d9083387 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.756260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 162/319] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:34.756922 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f9541-7480-4dfb-bb89-5fbcd8898ead None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.768276 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.768963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.769238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.769502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.770620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.772211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.774128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 158/320] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.779711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.802551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.882969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f9541-7480-4dfb-bb89-5fbcd8898ead None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.887995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:34.920499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8912-0123-4fd1-a939-014c54cddc02 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:34.922608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.923388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.924392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.924392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.924938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] RBAC: Adding query filter params (group.id=7f2ad812715647ff95a69ebab232b279, role.id=84eb2fe76ba84dfab84d2d1ab664754f, scope.project.id=66ff1cdc507747e5a54adbea719b6323) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:34.925330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.926939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.933545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 163/321] 10.222.0.36 () {56 vars in 1376 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/role_assignments?group.id=7f2ad812715647ff95a69ebab232b279&scope.project.id=66ff1cdc507747e5a54adbea719b6323&role.id=84eb2fe76ba84dfab84d2d1ab664754f => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.934458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:34.964767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:34.965815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:34.966242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:34.967084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:34.968352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] RBAC: Adding query filter params (group.id=7f2ad812715647ff95a69ebab232b279, role.id=ca9ae3dc2d9b42c0b35eca1fda566496, scope.project.id=66ff1cdc507747e5a54adbea719b6323) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:34.968829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:34.971385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:34.980560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 159/322] 10.222.0.36 () {56 vars in 1376 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/role_assignments?group.id=7f2ad812715647ff95a69ebab232b279&scope.project.id=66ff1cdc507747e5a54adbea719b6323&role.id=ca9ae3dc2d9b42c0b35eca1fda566496 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:34.981667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.100578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:35.119201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e399be-439a-4e84-8ffb-d6d1ee58ccde None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.130950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8017d-e6bf-45b8-b8d1-5590e01f5e2c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.168093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.169348 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.169782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.170201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.187973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:35.192917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:35.193457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:35.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:35.195133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:35.195765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.195765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.195765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.196042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 160/323] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:35.196042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.197854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:35.198594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:35.200615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 164/324] 10.222.0.36 () {56 vars in 1180 bytes} [Mon Oct 17 06:50:34 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:35.203283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.251652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:35.327411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75858076-86b2-4c05-8a6c-bd2866eb66c0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.336723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44039fd3-248c-452b-994d-bdd9b983ba7d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.371393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.371956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.372226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.372491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.377848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.378199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.378419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.378690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.379619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:35.379923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:35.381679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:35.383959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:35.389303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:35.390196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:35.391384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 161/325] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:35.392338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 165/326] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:35.396127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.396247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.398891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:35.508648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:35.512639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0901-ee22-43b3-b620-3aa01f532d6c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.512763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b651-0513-4bfb-bc88-d2fc30df4178 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.550674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.551546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.551980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.552249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.552969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] RBAC: Adding query filter params (group.id=7f2ad812715647ff95a69ebab232b279, role.id=84eb2fe76ba84dfab84d2d1ab664754f, scope.project.id=a6d647839334404c995bca069f96e0db) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:35.553272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:35.553461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.554110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.554391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.554659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.554749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:35.555348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:35.555650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:35.557051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:35.560974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 162/327] 10.222.0.36 () {56 vars in 1376 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/role_assignments?group.id=7f2ad812715647ff95a69ebab232b279&scope.project.id=a6d647839334404c995bca069f96e0db&role.id=84eb2fe76ba84dfab84d2d1ab664754f => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:35.561600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.566826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 166/328] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:35.574277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.655655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:35.691872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e580014-305f-4e36-be90-c5c6fcd1d515 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.695963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f564a95-7604-43ac-935e-a1ed9695fd89 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:35.727495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.728642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.728926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.729206 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.736243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:35.737176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:35.737663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:35.737931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:35.764412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:35.816143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:35.816757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:35.817629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:35.818216 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:35.894045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:35.932152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 167/329] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 06:50:35.935044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 163/330] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:35.938327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.941126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:35.999760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:36.040189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44aff3-458f-4b29-908e-51517f80798a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:36.040370 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b91c37-82c4-4be6-a753-13f5327595ad None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:36.072766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:36.073324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:36.073456 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:36.073634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:36.073869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] PATH_INFO: `/v3/projects/86c1b532428347929d42c84633169ba2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:36.073960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:36.074205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:36.074483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] PATH_INFO: `/v3/projects/a6d647839334404c995bca069f96e0db` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:36.075027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] RBAC: Authorizing `identity:get_project(project_id=86c1b532428347929d42c84633169ba2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:36.075614 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] RBAC: Authorizing `identity:get_project(project_id=a6d647839334404c995bca069f96e0db)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:36.076522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:36.077044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:36.078148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 164/331] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/projects/86c1b532428347929d42c84633169ba2 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:36.082471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 168/332] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:50:35 2022] GET /identity/v3/projects/a6d647839334404c995bca069f96e0db => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:36.083826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:36.083932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:36.117222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:36.181803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55823d8-89b1-48ec-8783-032200a5d32b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:36.188747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac342d-ccab-43c3-8e08-d2394ab7615d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:36.208244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:36.217347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:36.217970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:36.218245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:36.218520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] PATH_INFO: `/v3/groups/7f2ad812715647ff95a69ebab232b279` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:36.219629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] RBAC: Authorizing `identity:get_group(group_id=7f2ad812715647ff95a69ebab232b279)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:36.221064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:36.222105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:36.222625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 169/333] 10.222.0.36 () {56 vars in 1180 bytes} [Mon Oct 17 06:50:36 2022] GET /identity/v3/groups/7f2ad812715647ff95a69ebab232b279 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:36.222753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:36.223028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:36.223333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] PATH_INFO: `/v3/groups/7246ee5a489a4a2c99e935c723e436a6` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:36.224494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] RBAC: Authorizing `identity:get_group(group_id=7246ee5a489a4a2c99e935c723e436a6)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:36.225994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:36.227579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 165/334] 10.222.0.36 () {56 vars in 1180 bytes} [Mon Oct 17 06:50:36 2022] GET /identity/v3/groups/7246ee5a489a4a2c99e935c723e436a6 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:36.228556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:36.232944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:36.323017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06af822-3f48-4c41-875c-d424eb79451b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:36.328930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8915e-eb7f-41ba-846a-d30e12910ea8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:36.330389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:36.356013 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42bee6-fb27-4d0c-a7af-0c5674f271db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:36.356598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42bee6-fb27-4d0c-a7af-0c5674f271db None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:36.356873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42bee6-fb27-4d0c-a7af-0c5674f271db None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:36.357139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42bee6-fb27-4d0c-a7af-0c5674f271db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:36.357878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42bee6-fb27-4d0c-a7af-0c5674f271db None admin] RBAC: Adding query filter params (group.id=7f2ad812715647ff95a69ebab232b279, role.id=ca9ae3dc2d9b42c0b35eca1fda566496, scope.project.id=a6d647839334404c995bca069f96e0db) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:36.358180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42bee6-fb27-4d0c-a7af-0c5674f271db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:36.359633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42bee6-fb27-4d0c-a7af-0c5674f271db None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:36.363355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cc9a6-b498-44a4-9cf1-cb34405e26db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:36.363945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cc9a6-b498-44a4-9cf1-cb34405e26db None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:36.364220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cc9a6-b498-44a4-9cf1-cb34405e26db None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:36.364488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cc9a6-b498-44a4-9cf1-cb34405e26db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:36.365162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 170/335] 10.222.0.36 () {56 vars in 1376 bytes} [Mon Oct 17 06:50:36 2022] GET /identity/v3/role_assignments?group.id=7f2ad812715647ff95a69ebab232b279&scope.project.id=a6d647839334404c995bca069f96e0db&role.id=ca9ae3dc2d9b42c0b35eca1fda566496 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:36.365264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cc9a6-b498-44a4-9cf1-cb34405e26db None admin] RBAC: Adding query filter params (group.id=7246ee5a489a4a2c99e935c723e436a6, role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=86c1b532428347929d42c84633169ba2) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:36.365490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cc9a6-b498-44a4-9cf1-cb34405e26db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:36.366981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cc9a6-b498-44a4-9cf1-cb34405e26db None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:36.373094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 166/336] 10.222.0.36 () {56 vars in 1376 bytes} [Mon Oct 17 06:50:36 2022] GET /identity/v3/role_assignments?group.id=7246ee5a489a4a2c99e935c723e436a6&scope.project.id=86c1b532428347929d42c84633169ba2&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:38.994675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668123-8998-4d3c-8020-f4e7cecb506d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:38.995189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668123-8998-4d3c-8020-f4e7cecb506d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:38.995189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668123-8998-4d3c-8020-f4e7cecb506d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:38.996848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 171/337] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:39.001921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1ac7f-5d0a-4aca-b80b-50f2e4d8da21 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.002333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1ac7f-5d0a-4aca-b80b-50f2e4d8da21 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.002505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1ac7f-5d0a-4aca-b80b-50f2e4d8da21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.023811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a8e1ac7f-5d0a-4aca-b80b-50f2e4d8da21 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:39.025519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a8e1ac7f-5d0a-4aca-b80b-50f2e4d8da21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:39.110151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 167/338] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:39.239367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8293e57-17bf-4d12-b709-14059ec5ac17 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.239690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8293e57-17bf-4d12-b709-14059ec5ac17 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.239906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8293e57-17bf-4d12-b709-14059ec5ac17 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.241239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 172/339] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:39.246538 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a8df-4a35-4fba-89df-7ad00d1f720f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.246764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a8df-4a35-4fba-89df-7ad00d1f720f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.247016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a8df-4a35-4fba-89df-7ad00d1f720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.266410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ca32a8df-4a35-4fba-89df-7ad00d1f720f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:39.267694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ca32a8df-4a35-4fba-89df-7ad00d1f720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:39.360996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 168/340] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:39.448415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8293e57-17bf-4d12-b709-14059ec5ac17 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:39.448844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62889018-006f-4da4-8240-9f839a4f50c6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.449126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62889018-006f-4da4-8240-9f839a4f50c6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.449399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62889018-006f-4da4-8240-9f839a4f50c6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.450988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 169/341] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:39.456092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9c29-e303-4053-a956-9bbe4346aa5c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.456368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9c29-e303-4053-a956-9bbe4346aa5c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.456640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9c29-e303-4053-a956-9bbe4346aa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.482543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6edd9c29-e303-4053-a956-9bbe4346aa5c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:39.483729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6edd9c29-e303-4053-a956-9bbe4346aa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:39.522213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:39.526703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:39.585107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8293e57-17bf-4d12-b709-14059ec5ac17 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:39.623243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 170/342] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:39.636500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:39.637117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.637388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.637657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.647976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:39.649399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:39.663714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:39.664837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:39.665747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:39.665747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:39.665747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:39.667814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 173/343] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:39.670211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9c29-e303-4053-a956-9bbe4346aa5c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:39.675098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:39.802312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87659214-7565-44cb-86e5-dfd7932b6138 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:39.817999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:39.848340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb153040-89f1-4610-8872-386320597027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:39.850833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9c29-e303-4053-a956-9bbe4346aa5c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:39.852425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb153040-89f1-4610-8872-386320597027 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.853031 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb153040-89f1-4610-8872-386320597027 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.854919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb153040-89f1-4610-8872-386320597027 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.855618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb153040-89f1-4610-8872-386320597027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:39.855930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb153040-89f1-4610-8872-386320597027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:39.857319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb153040-89f1-4610-8872-386320597027 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:39.884763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 174/344] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:39.893517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb153040-89f1-4610-8872-386320597027 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:39.916977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:39.917615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:39.917893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:39.918160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:39.933401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:39.934863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:39.952579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:39.953492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:39.954343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:39.954343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:39.954343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:39.954343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 171/345] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:39.954964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:39.970283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:40.030273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb153040-89f1-4610-8872-386320597027 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:40.069617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:40.070295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.070672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.071041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.072391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:40.074073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:40.086178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a1535-9898-4162-b9f2-2f875426e710 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:40.097244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 175/346] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:39 2022] POST /identity/v3/users => generated 246 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:50:40.098302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:40.123857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:40.124508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.124780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.125053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.133266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:40.138756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:40.141142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:40.157267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:40.158756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:40.159715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:40.159715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:40.159715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:40.159715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 172/347] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:40.165225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:40.234036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:40.234794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e992f3d-7bf6-4ef0-951f-2a0b39db4531 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:40.277430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6700c7-7427-41e0-bf6a-168ae00b285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:40.278168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6700c7-7427-41e0-bf6a-168ae00b285d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.278455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6700c7-7427-41e0-bf6a-168ae00b285d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.278768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6700c7-7427-41e0-bf6a-168ae00b285d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.279537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6700c7-7427-41e0-bf6a-168ae00b285d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:40.279839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6700c7-7427-41e0-bf6a-168ae00b285d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:40.280357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031719c-6b91-4b72-97d1-a20186c7ddab None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:40.281261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6700c7-7427-41e0-bf6a-168ae00b285d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:40.295697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 176/348] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:40.301558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47357ef-4787-44f4-b0aa-08404c0ac9e5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.301839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47357ef-4787-44f4-b0aa-08404c0ac9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.302100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47357ef-4787-44f4-b0aa-08404c0ac9e5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.303902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 177/349] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:40.304661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47357ef-4787-44f4-b0aa-08404c0ac9e5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:40.319177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58083354-532b-4357-99ae-6fbbe98d50a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:40.319861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083354-532b-4357-99ae-6fbbe98d50a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.320077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083354-532b-4357-99ae-6fbbe98d50a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.320345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083354-532b-4357-99ae-6fbbe98d50a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.321083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58083354-532b-4357-99ae-6fbbe98d50a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:40.321385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58083354-532b-4357-99ae-6fbbe98d50a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:40.322902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58083354-532b-4357-99ae-6fbbe98d50a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:40.337668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 173/350] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:40.339024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530b777-2473-46a0-8e80-22deb8735f59 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.339341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530b777-2473-46a0-8e80-22deb8735f59 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.339618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530b777-2473-46a0-8e80-22deb8735f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.361590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7530b777-2473-46a0-8e80-22deb8735f59 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:40.363371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7530b777-2473-46a0-8e80-22deb8735f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:40.411296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:40.425888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47357ef-4787-44f4-b0aa-08404c0ac9e5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:40.455504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 174/351] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:40.456869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530b777-2473-46a0-8e80-22deb8735f59 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:40.462693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54932975-0965-4cca-9012-db109bff055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:40.463230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54932975-0965-4cca-9012-db109bff055a None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.463518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54932975-0965-4cca-9012-db109bff055a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.463790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54932975-0965-4cca-9012-db109bff055a None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.464430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54932975-0965-4cca-9012-db109bff055a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:40.465863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54932975-0965-4cca-9012-db109bff055a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:40.517166 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 178/352] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:40 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:40.531585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:40.602997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530b777-2473-46a0-8e80-22deb8735f59 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:40.639810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afb8e4-73aa-41c4-9031-a20994fa2bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:40.640418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb8e4-73aa-41c4-9031-a20994fa2bf3 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.640694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb8e4-73aa-41c4-9031-a20994fa2bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.640964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb8e4-73aa-41c4-9031-a20994fa2bf3 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.641610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afb8e4-73aa-41c4-9031-a20994fa2bf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:40.643061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afb8e4-73aa-41c4-9031-a20994fa2bf3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:40.675873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 175/353] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:40 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:40.836451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54932975-0965-4cca-9012-db109bff055a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:40.905217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f29a5a-4d9e-4b03-9aa8-bda4fab384d8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.905843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f29a5a-4d9e-4b03-9aa8-bda4fab384d8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.906331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f29a5a-4d9e-4b03-9aa8-bda4fab384d8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.910146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 176/354] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:40.913585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61236-ca6d-4e85-b05d-cf13609c2c22 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:40.914112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61236-ca6d-4e85-b05d-cf13609c2c22 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:40.914581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61236-ca6d-4e85-b05d-cf13609c2c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:40.946179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-28b61236-ca6d-4e85-b05d-cf13609c2c22 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:40.946179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-28b61236-ca6d-4e85-b05d-cf13609c2c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:40.965493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54932975-0965-4cca-9012-db109bff055a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:41.006431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:41.007068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:41.007377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:41.007649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:41.020420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:41.021873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:41.027957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:41.031799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 177/355] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:41.044451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:41.045889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:41.046938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:41.046938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:41.046938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:41.048378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 179/356] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:41.054388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b61236-ca6d-4e85-b05d-cf13609c2c22 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:41.090379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:41.198440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b61236-ca6d-4e85-b05d-cf13609c2c22 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:41.246757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:41.247492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:41.247771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:41.248046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:41.248694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:41.248995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:41.250373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:41.262532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 178/357] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:41.268085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:41.339539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:41.430489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4691b6-7bc0-4d0c-b303-48f17ebe5963 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:41.468919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:41.470022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:41.470499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:41.470964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:41.471814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:41.473777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:41.497460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 180/358] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:41 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:41.553240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:41.565535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5254bd-8242-4dec-967d-05d5c826ab5c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:41.610266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:41.610965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:41.611288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:41.611571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:41.612277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:41.613771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:41.622086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.sql.core [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 06:50:41.622086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 06:50:41.622086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [parameters: {'id': '4f9d9a8d20b34712ad4bc0e3683032cd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 06:50:41.622086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81488) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:50:41.629920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return ref.to_dict() Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:50:41.630876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application yield resource Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise self.value Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application session.commit() Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:50:41.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:50:41.633356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:50:41.634762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:50:41.636669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [parameters: {'id': '4f9d9a8d20b34712ad4bc0e3683032cd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:41.638362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 06:50:41.640269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:41.649197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 179/359] 10.222.0.36 () {60 vars in 1131 bytes} [Mon Oct 17 06:50:41 2022] POST /identity/v3/roles => generated 144 bytes in 292 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:41.649345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:41.745582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:41.794634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac5ed7-f713-4ca5-af39-9342cad489cf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:41.829539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03e4c2-c0db-4832-a12d-a371f250af5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:41.830219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03e4c2-c0db-4832-a12d-a371f250af5c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:41.830495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03e4c2-c0db-4832-a12d-a371f250af5c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:41.830761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03e4c2-c0db-4832-a12d-a371f250af5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:41.841404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03e4c2-c0db-4832-a12d-a371f250af5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fb03e4c2-c0db-4832-a12d-a371f250af5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:41.847369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:41.848263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:41.849144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 181/360] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:50:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:41.854656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:41.968331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bafc77-38b8-4d62-931c-27cdc0354900 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:42.004163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fcd56-ce7b-4759-9c34-1ab347ece43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:42.004685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fcd56-ce7b-4759-9c34-1ab347ece43e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:42.004954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fcd56-ce7b-4759-9c34-1ab347ece43e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:42.005216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fcd56-ce7b-4759-9c34-1ab347ece43e None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:42.005867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fcd56-ce7b-4759-9c34-1ab347ece43e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:42.006162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fcd56-ce7b-4759-9c34-1ab347ece43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:42.007600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fcd56-ce7b-4759-9c34-1ab347ece43e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:42.016257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 180/361] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:42.604768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b22cb-836c-491c-85ce-16dbfd738933 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:42.605302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b22cb-836c-491c-85ce-16dbfd738933 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:42.605302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b22cb-836c-491c-85ce-16dbfd738933 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:42.606885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 182/362] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:42.612216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679370-8e42-4f26-92e6-40811692cfb8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:42.612624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679370-8e42-4f26-92e6-40811692cfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:42.612806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679370-8e42-4f26-92e6-40811692cfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:42.636194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-68679370-8e42-4f26-92e6-40811692cfb8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:42.637380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-68679370-8e42-4f26-92e6-40811692cfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:42.655609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:42.713344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 181/363] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:43.039008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b22cb-836c-491c-85ce-16dbfd738933 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:43.176839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b22cb-836c-491c-85ce-16dbfd738933 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:43.193978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:43.224562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf254a5-e929-4528-8e92-14729a55f36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:43.225224 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf254a5-e929-4528-8e92-14729a55f36f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.225499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf254a5-e929-4528-8e92-14729a55f36f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.225766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf254a5-e929-4528-8e92-14729a55f36f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.248617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf254a5-e929-4528-8e92-14729a55f36f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3cf254a5-e929-4528-8e92-14729a55f36f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:43.253368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:43.254312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:43.255093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:43.257241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 183/364] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:43.265476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68679370-8e42-4f26-92e6-40811692cfb8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:43.379364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce6b64-cb8f-4663-92a4-96ee20760463 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.379547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce6b64-cb8f-4663-92a4-96ee20760463 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.379805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce6b64-cb8f-4663-92a4-96ee20760463 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.381339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 184/365] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:43.387063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257044e-e8a1-4cc7-8453-500726d09934 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.387384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257044e-e8a1-4cc7-8453-500726d09934 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.387636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257044e-e8a1-4cc7-8453-500726d09934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.420174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8257044e-e8a1-4cc7-8453-500726d09934 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:43.423399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8257044e-e8a1-4cc7-8453-500726d09934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:43.442879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68679370-8e42-4f26-92e6-40811692cfb8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:43.484051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:43.484674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.484953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.485225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.485888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:43.486191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:43.487594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:43.506695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 182/366] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:43.512160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:43.519030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 185/367] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:43.520428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414bb8f-7fee-4222-ace2-06437cc50510 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.520714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414bb8f-7fee-4222-ace2-06437cc50510 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.520982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414bb8f-7fee-4222-ace2-06437cc50510 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.522217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 186/368] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:43.528143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfddde6-34ba-4395-9147-2ba25e06c040 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.528425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfddde6-34ba-4395-9147-2ba25e06c040 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.528702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfddde6-34ba-4395-9147-2ba25e06c040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.534501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:43.548809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9dfddde6-34ba-4395-9147-2ba25e06c040 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:43.549990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9dfddde6-34ba-4395-9147-2ba25e06c040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:43.657572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 187/369] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:43.695861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a8de-63fd-46d1-862d-7c2a19fa990c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:43.741678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:43.742307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.742583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.742854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.753528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:43.754982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:43.769636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:43.770530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:43.771413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:43.771413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:43.771413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:43.771413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 183/370] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:43.777656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfddde6-34ba-4395-9147-2ba25e06c040 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:43.809069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:43.911086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfddde6-34ba-4395-9147-2ba25e06c040 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:43.948017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:43.948627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:43.948906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:43.949182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:43.949826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:43.950130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:43.951549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:43.968425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:43.972115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 188/371] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:43.979252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.048777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.096945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056b43-a796-44f6-8aac-48bc59caaa8b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.098755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.109714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb03c-df42-4b47-bb5e-44acb1ba9c9e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.139953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.140581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.141144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.141443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.153119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:44.158840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:44.159771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:44.160575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 184/372] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:44.161278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.166207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.166207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.166207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.166207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.178607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.181201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:44.196234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:44.197278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:50:44.198190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:44.205561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 189/373] 10.222.0.36 () {56 vars in 1093 bytes} [Mon Oct 17 06:50:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:44.206877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.287292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbaca8-c380-41f9-9d96-2b89f9a039d4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.314649 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.317024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7b91d-fc6d-4211-8630-3f5f5775a05b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.327284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.327919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.328195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.328467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.339952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:44.346048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:44.346957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:44.347814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 185/374] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:44.348473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.358384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11798d9c-bdd2-4d14-8b0e-f42b54d153e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.359429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798d9c-bdd2-4d14-8b0e-f42b54d153e1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.359905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798d9c-bdd2-4d14-8b0e-f42b54d153e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.360360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798d9c-bdd2-4d14-8b0e-f42b54d153e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.361016 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11798d9c-bdd2-4d14-8b0e-f42b54d153e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:44.361316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11798d9c-bdd2-4d14-8b0e-f42b54d153e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.362879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11798d9c-bdd2-4d14-8b0e-f42b54d153e1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.376364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 190/375] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:44.376364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667863b1-f196-4500-940a-88adb8d9be20 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.376364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667863b1-f196-4500-940a-88adb8d9be20 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.376364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667863b1-f196-4500-940a-88adb8d9be20 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.376364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 191/376] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:44.376364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667863b1-f196-4500-940a-88adb8d9be20 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.393535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.456030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a789cbe-f687-4640-b6ac-1161faab870c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.488959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c446a-6652-45f9-b1a4-a23ce3d65cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.489558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c446a-6652-45f9-b1a4-a23ce3d65cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.489798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667863b1-f196-4500-940a-88adb8d9be20 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.489911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c446a-6652-45f9-b1a4-a23ce3d65cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.490098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c446a-6652-45f9-b1a4-a23ce3d65cf2 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.490812 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c446a-6652-45f9-b1a4-a23ce3d65cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:44.491118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c446a-6652-45f9-b1a4-a23ce3d65cf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.492577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c446a-6652-45f9-b1a4-a23ce3d65cf2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.526508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.527561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.528016 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.528492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.528924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 186/377] 10.222.0.36 () {56 vars in 1133 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:44.529507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:44.529964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.530137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808353-b001-4116-ab71-863bebadd124 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.530421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808353-b001-4116-ab71-863bebadd124 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.530692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808353-b001-4116-ab71-863bebadd124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.532083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.543492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 192/378] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:44.544650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.552330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bb808353-b001-4116-ab71-863bebadd124 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:44.553510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb808353-b001-4116-ab71-863bebadd124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:44.593649 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.631236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 187/379] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:44.631236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb808353-b001-4116-ab71-863bebadd124 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.639734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.669126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313f4e-d7c9-4ca0-816e-04930676f3d7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.716068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.716621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.716985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.717270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.730683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.732940 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.737043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb808353-b001-4116-ab71-863bebadd124 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:44.751305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:44.752217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:44.753233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:44.753233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:44.753233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:44.753233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 193/380] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:44.754437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.781656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.782801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.783076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.783426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.799485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.801252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:44.823078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:44.824944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:44.826861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:44.826861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:44.826861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:44.826861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 188/381] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:44.826861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.862471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.882956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3e196-a77d-4c12-a23a-6f60eb6f3cd7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.912646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:44.923325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.923954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.924231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.924580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.935859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.937561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.944606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987af66e-c84f-4622-bb48-3959917b7145 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:44.951737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:44.955390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:44.956400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:44.956400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:44.956400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:44.956400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 194/382] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:44.956400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:44.980359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb4e32-06e7-47ec-9551-5131f48fc323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:44.980886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb4e32-06e7-47ec-9551-5131f48fc323 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.981162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb4e32-06e7-47ec-9551-5131f48fc323 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.981429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb4e32-06e7-47ec-9551-5131f48fc323 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.982057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb4e32-06e7-47ec-9551-5131f48fc323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:44.982360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb4e32-06e7-47ec-9551-5131f48fc323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:44.983781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb4e32-06e7-47ec-9551-5131f48fc323 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:44.995405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 189/383] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:44.996559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e045-40c0-488d-bf86-557533599fa4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:44.996836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e045-40c0-488d-bf86-557533599fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:44.997122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e045-40c0-488d-bf86-557533599fa4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:44.998289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 190/384] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:44.998968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e045-40c0-488d-bf86-557533599fa4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.058081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ea8fd-93f1-4f5d-a43e-c8d109d8433a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.080678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.091594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.092194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.092466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.092731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.093352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:45.093644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.095012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.096547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e045-40c0-488d-bf86-557533599fa4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.106256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 195/385] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:45.107085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.129640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.130253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.130524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.130792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.140416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:45.145785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:45.146746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:45.147544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:45.151506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.151634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 191/386] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:45.152579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.208388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241cac3-0010-4275-bba0-f9f99ddd9695 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.241468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964ade8-059e-45ce-8d1b-7c7e2c9f1796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.242000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964ade8-059e-45ce-8d1b-7c7e2c9f1796 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.242273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964ade8-059e-45ce-8d1b-7c7e2c9f1796 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.242539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964ade8-059e-45ce-8d1b-7c7e2c9f1796 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.243773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964ade8-059e-45ce-8d1b-7c7e2c9f1796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:45.244071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964ade8-059e-45ce-8d1b-7c7e2c9f1796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.245423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964ade8-059e-45ce-8d1b-7c7e2c9f1796 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.249786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441237a-2c1e-425a-a901-4d39f60937fa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.256150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 196/387] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:45.257268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179666cb-a694-4fc4-93a9-7eef137f3ce6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.257543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179666cb-a694-4fc4-93a9-7eef137f3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.257816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179666cb-a694-4fc4-93a9-7eef137f3ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.274764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-179666cb-a694-4fc4-93a9-7eef137f3ce6 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:45.275970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-179666cb-a694-4fc4-93a9-7eef137f3ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:45.286407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.287017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.287318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.287609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.294929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.296393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:45.308804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:45.309710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:45.310586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 192/388] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:45.311129 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.320296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.345012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 197/389] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:45.346054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179666cb-a694-4fc4-93a9-7eef137f3ce6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.387001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.416224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24368e6-3bac-4357-b61b-e5d1f8dc9572 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.450749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.451402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.451685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.451960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.458854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.460351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.461464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179666cb-a694-4fc4-93a9-7eef137f3ce6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:45.471825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:45.472898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 193/390] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:45.473750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.507389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.508214 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.509500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.510776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.511611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:45.512099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.513593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.522390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 198/391] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:45.527952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.541674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.578444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da07730-d3fe-4dae-9691-240b2a373e56 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.612494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.613117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.613390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.613659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.621556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.623002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.625134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.634018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7b99-5fd4-4b73-80f8-ace8188ade17 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:45.634143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:45.635132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:45.636080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 194/392] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:45.636660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.671602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.672193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.672464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.672726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.673429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:45.673727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.675155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.710204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 199/393] 10.222.0.36 () {56 vars in 1137 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:45.710996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.735905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08271ff8-fd0d-4e64-83a0-1b43175a9ad8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.760524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.769040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.769635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.769910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.770178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.770852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:45.771149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.772601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.789778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 195/394] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:45.790627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.815069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c221-6e3c-4a0d-a470-d5c63cb96207 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.848616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.849226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.849497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.849763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.859415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.860866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:45.875469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:45.876546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:45.877412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:45.877412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:45.877412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:45.877412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 200/395] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:45.877412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.879799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:45.892460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ce113-7a9b-4683-9d2e-db9212fd44be None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.925400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:45.926024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:45.926304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:45.926648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:45.938592 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:45.940085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:45.954746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:45.955785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:45.956631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:45.956631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:45.956631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:45.956631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 196/396] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:45.956839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:45.975662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c4199-6e09-42f5-8c5f-192a684ca994 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:45.997591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.009361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.009984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.010258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.010579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.011269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:46.011574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.013002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.029370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 201/397] 10.222.0.36 () {56 vars in 1137 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:46.030131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.056808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1940ed7-360e-4af6-978e-1867dce8a54a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.087246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.090300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.090914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.091187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.091481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.101345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.103359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:46.118494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:46.119512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:46.120363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:46.120363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:46.120363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:46.120363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 197/398] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:46.120363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.130515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c861f-bc4d-4c83-b576-8c29039d406d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.168056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.168714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.168992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.169260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/74a91af405e24a0b8e9c893e1c299db4/roles/f2f75648d0384a4cac1041775239be27` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.183399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=74a91af405e24a0b8e9c893e1c299db4, role_id=f2f75648d0384a4cac1041775239be27)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.184841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.198703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 202/399] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:46 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/74a91af405e24a0b8e9c893e1c299db4/roles/f2f75648d0384a4cac1041775239be27 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:46.199298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.215027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.228994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edba26-e9d2-4a96-936e-e46139b31491 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.266711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.267290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.267563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.267834 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.268485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:46.268788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.270193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.287744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 198/400] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:46.287744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.319691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab7203-f88c-4192-a916-8a2a3c76bc7a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.332156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.365333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.365333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.365333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.365681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.366681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:46.366990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.368983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.382271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 203/401] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:46.383425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.393768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5b8a6-3d48-4925-a5d4-ff0624bc9303 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.430148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.431337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.431613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.431878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.442037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.443506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:46.457893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:46.458813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:46.459671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:46.459671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:46.459671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:46.459671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 199/402] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:46.460428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.466750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.500682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de8bb1-ca98-467f-b38a-e188e334bcf4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.537168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.537696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.537971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.538239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.538891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:46.539186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.540848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.554573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 204/403] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:46.556302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.579763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bf58f-e63e-46f3-8880-9a1fb7a6be39 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.588326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.614593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.615197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.615507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.615779 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.624232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.625692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:46.637171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:46.638082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 200/404] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:46.639615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.669257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334305d-52a8-46db-b1b0-0d11b84d0e4a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.706906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.707569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.707569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.707904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.708470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.709882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.713334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.733729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 205/405] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:46 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:46.734974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.769397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3354821-01ce-4200-bd73-01a47dc0e870 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.817713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.818252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.818531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.818800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.819529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:46.820473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.822285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.837051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 201/406] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:46.839249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.849859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:46.870693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b60b62-99b4-4010-b42b-9f0627f40431 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.911494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:46.912568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:46.912851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:46.913121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:46.922962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:46.924460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:46.939091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:46.940068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:46.940904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 206/407] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:46.941359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:46.970453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3fcb-2566-46c3-90dc-32bad634820b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:46.997066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:47.019949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.020501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.020779 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.021052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.021777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:47.022083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.023627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.046562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 202/408] 10.222.0.36 () {56 vars in 1139 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:47.046562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:47.083545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d6b87-9e71-426f-835a-623bfbd521c3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.130416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.131594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.132066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.132720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.141955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.144349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.160561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c0a97-bb34-45f5-8b4c-751350b89b74 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:47.162311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:47.163189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:47.164074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:47.164465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 207/409] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:47.165872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:47.176594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:47.210533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.211509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.212513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.213275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.214102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:47.214835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.216730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.231833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 203/410] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:47.232839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:47.283236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078d2e-01bf-4b93-aeb9-1312de88a0f5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.287292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:47.316480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.317259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.317641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.318051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.330475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.332309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:47.348378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:47.349476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:47.350518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:47.350518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:47.350518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:47.351719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 208/411] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:47.352718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:47.363349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ec29f-1043-40b0-ad6d-0ff4f2f1b757 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.401433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.402151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.402499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.402847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.403660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:47.404039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.405607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.418450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 204/412] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:47.419537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:47.427646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:47.464214 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78e12e-7385-4957-9a6d-b7007e4aa6f9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.506037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.506749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.507029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.507342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/a0a3956bb98749e6bb19ee7f3465eac9/roles/f2f75648d0384a4cac1041775239be27` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.523054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:47.524103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=a0a3956bb98749e6bb19ee7f3465eac9, role_id=f2f75648d0384a4cac1041775239be27)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.526147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.527622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f233-eddb-49b9-a8f5-d405bfe5f20b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.544666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 209/413] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:47 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/a0a3956bb98749e6bb19ee7f3465eac9/roles/f2f75648d0384a4cac1041775239be27 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:47.545376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:47.566034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.566702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.566983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.567341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.567998 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:47.568354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.569840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.583799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 205/414] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:47.595240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:47.710435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf710d9-ad57-435a-9933-f9a9c9fcc822 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.731649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:47.777344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.778145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.778420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.778713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/92c66f0a106f4c8e99ac7075d76767d4/roles/f2f75648d0384a4cac1041775239be27` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.782613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594eb8-d4f6-4502-93ef-18a3fa97954b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:47.798158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=92c66f0a106f4c8e99ac7075d76767d4, role_id=f2f75648d0384a4cac1041775239be27)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.799911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.817633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 210/415] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:47 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/92c66f0a106f4c8e99ac7075d76767d4/roles/f2f75648d0384a4cac1041775239be27 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:47.837808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:47.838575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:47.838857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:47.839129 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:47.847064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:47.848745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:47.862604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:47.863530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:47.864377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 206/416] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:47.871770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:48.009639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911998a5-589b-4e0c-a716-52588837b257 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:48.054431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e4b09-7cb4-419c-8fb2-4bef63a55566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:48.055062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4b09-7cb4-419c-8fb2-4bef63a55566 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:48.055364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4b09-7cb4-419c-8fb2-4bef63a55566 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:48.055770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4b09-7cb4-419c-8fb2-4bef63a55566 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:48.056470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4b09-7cb4-419c-8fb2-4bef63a55566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:48.056770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4b09-7cb4-419c-8fb2-4bef63a55566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:48.058212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4b09-7cb4-419c-8fb2-4bef63a55566 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:48.082842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 211/417] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:48.089143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:48.124769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:48.240022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea7630-669d-4ca7-ad0c-99135cbed077 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:48.293666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0d169-ef73-415a-9d94-a806ce41033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:48.294268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0d169-ef73-415a-9d94-a806ce41033b None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:48.294546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0d169-ef73-415a-9d94-a806ce41033b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:48.294816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0d169-ef73-415a-9d94-a806ce41033b None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/56967a5531ea4e308af5b89c13524c14/roles/f2f75648d0384a4cac1041775239be27` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:48.314434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0d169-ef73-415a-9d94-a806ce41033b None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=56967a5531ea4e308af5b89c13524c14, role_id=f2f75648d0384a4cac1041775239be27)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:48.315946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0d169-ef73-415a-9d94-a806ce41033b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:48.337343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 207/418] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:48 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/56967a5531ea4e308af5b89c13524c14/roles/f2f75648d0384a4cac1041775239be27 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:48.517126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207843-7566-465f-9a32-07803fa4d45c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:48.517503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207843-7566-465f-9a32-07803fa4d45c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:48.517677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207843-7566-465f-9a32-07803fa4d45c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:48.519379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 212/419] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:48.537067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b5ce7-5257-449f-85a9-d8f4fc6d6e46 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:48.537371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b5ce7-5257-449f-85a9-d8f4fc6d6e46 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:48.537620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b5ce7-5257-449f-85a9-d8f4fc6d6e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:48.564316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a56b5ce7-5257-449f-85a9-d8f4fc6d6e46 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:48.565446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a56b5ce7-5257-449f-85a9-d8f4fc6d6e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:48.637622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:48.644039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 208/420] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:48.955695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207843-7566-465f-9a32-07803fa4d45c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:48.975107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:49.068002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207843-7566-465f-9a32-07803fa4d45c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:49.117790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97217545-d491-4aef-9f49-04a5bc28ec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:49.119012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97217545-d491-4aef-9f49-04a5bc28ec3b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:49.119309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97217545-d491-4aef-9f49-04a5bc28ec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:49.119574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97217545-d491-4aef-9f49-04a5bc28ec3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:49.138168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97217545-d491-4aef-9f49-04a5bc28ec3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-97217545-d491-4aef-9f49-04a5bc28ec3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:49.142924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:49.143855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:49.144660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 213/421] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:49.150247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b5ce7-5257-449f-85a9-d8f4fc6d6e46 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:49.219324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060bdf-dcae-44fd-a90c-ddd7ee6a776b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:49.219528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060bdf-dcae-44fd-a90c-ddd7ee6a776b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:49.219774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060bdf-dcae-44fd-a90c-ddd7ee6a776b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:49.221346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 214/422] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:49.225783 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c44a5d-5a1d-4997-a4e7-c888b23fcab1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:49.226055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c44a5d-5a1d-4997-a4e7-c888b23fcab1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:49.226319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c44a5d-5a1d-4997-a4e7-c888b23fcab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:49.251359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-86c44a5d-5a1d-4997-a4e7-c888b23fcab1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:49.252541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-86c44a5d-5a1d-4997-a4e7-c888b23fcab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:49.307954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b5ce7-5257-449f-85a9-d8f4fc6d6e46 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:49.356150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:49.356760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:49.357039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:49.357310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:49.358087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:49.358269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:49.359687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:49.373526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 209/423] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:49.378199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 215/424] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:50:49.379448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:49.463429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:49.503768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3868d-a2aa-4879-aabb-67cb03e3712f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:49.538031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:49.538652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:49.538930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:49.539198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:49.550821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:49.552400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:49.567535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:49.568422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:49.569250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:49.569250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:49.569250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:49.569250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 210/425] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:49.574709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c44a5d-5a1d-4997-a4e7-c888b23fcab1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:49.589583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:49.690585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:49.691052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c44a5d-5a1d-4997-a4e7-c888b23fcab1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:49.726728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:49.727862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:49.728138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:49.728417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:49.729021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:49.729314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:49.730663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:49.742187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 216/426] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:49.747551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:49.833632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:49.851432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298162a9-1649-47de-86ac-f72f404b41ca None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:49.886338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:49.886958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:49.887259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:49.887536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:49.895225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:49.896712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:49.908766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:49.909669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:49.910565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 217/427] 10.222.0.36 () {56 vars in 1093 bytes} [Mon Oct 17 06:50:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:49.916043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:49.985163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea275bd-6fc5-4e38-84e6-9cfa17e4fe27 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.015565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:50.021130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f489fe2-f9be-4409-95d8-cb634f1d3754 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.056332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.056938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.057213 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.057479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.058196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:50.058490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:50.059966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:50.063610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:50.079319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 218/428] 10.222.0.36 () {56 vars in 1133 bytes} [Mon Oct 17 06:50:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:50.084885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:50.087917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.088567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.088845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.089113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.100794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:50.106249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:50.107384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:50.108384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:50.110946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 211/429] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:50.115608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:50.262083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c794efe-157f-44e3-b6e8-1df50fa98b9f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.284793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067417f7-420c-4424-9f61-55974c840ad2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.317800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.319729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.320012 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.320279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.321611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:50.323282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:50.327038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:50.347290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 212/430] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:50.349931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9c1bf-0c2e-473e-9c66-8c490ce5c1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.350528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9c1bf-0c2e-473e-9c66-8c490ce5c1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.350801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9c1bf-0c2e-473e-9c66-8c490ce5c1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.351064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9c1bf-0c2e-473e-9c66-8c490ce5c1b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.352637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:50.427735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:50.447604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:50.472387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9c1bf-0c2e-473e-9c66-8c490ce5c1b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:50.474101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9c1bf-0c2e-473e-9c66-8c490ce5c1b8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:50.520346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1894e4-371c-4394-a5c7-c10800cc419c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.568201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fc4c2-f778-4b6c-bb04-c78aca4985ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.569351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc4c2-f778-4b6c-bb04-c78aca4985ca None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.569745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc4c2-f778-4b6c-bb04-c78aca4985ca None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.570019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc4c2-f778-4b6c-bb04-c78aca4985ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.582674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fc4c2-f778-4b6c-bb04-c78aca4985ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:50.584150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fc4c2-f778-4b6c-bb04-c78aca4985ca None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-0a9fc4c2-f778-4b6c-bb04-c78aca4985ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:50.603991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:50.604904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:50.605741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:50.605741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:50.605741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:50.605741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 213/431] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:50.611232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5209a1-ca8d-43d0-a4e7-e74dc3e4a50d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.611232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5209a1-ca8d-43d0-a4e7-e74dc3e4a50d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.611232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5209a1-ca8d-43d0-a4e7-e74dc3e4a50d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.611232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 214/432] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:50.615007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5209a1-ca8d-43d0-a4e7-e74dc3e4a50d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:50.617538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 219/433] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:50.619268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c89ca-4240-48bf-84e2-81703557bf05 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.619694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c89ca-4240-48bf-84e2-81703557bf05 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.620087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c89ca-4240-48bf-84e2-81703557bf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.647697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7e8c89ca-4240-48bf-84e2-81703557bf05 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:50.649191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7e8c89ca-4240-48bf-84e2-81703557bf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:50.716207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:50.723958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 220/434] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:50.731011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5209a1-ca8d-43d0-a4e7-e74dc3e4a50d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.736105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c89ca-4240-48bf-84e2-81703557bf05 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:50.762735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:50.776643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737d6f9-1343-4c0f-b12d-feed292255da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.777261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737d6f9-1343-4c0f-b12d-feed292255da None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.777537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737d6f9-1343-4c0f-b12d-feed292255da None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.777792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737d6f9-1343-4c0f-b12d-feed292255da None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.778434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737d6f9-1343-4c0f-b12d-feed292255da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:50.778736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737d6f9-1343-4c0f-b12d-feed292255da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:50.780208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737d6f9-1343-4c0f-b12d-feed292255da None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:50.797092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 215/435] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:50.798414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8ffa-c11b-4fa2-bb9f-0087a337c37f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.798685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8ffa-c11b-4fa2-bb9f-0087a337c37f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.799011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8ffa-c11b-4fa2-bb9f-0087a337c37f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.804674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 216/436] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:50.804674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8ffa-c11b-4fa2-bb9f-0087a337c37f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:50.867250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c89ca-4240-48bf-84e2-81703557bf05 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.904821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a29bf-f730-481b-b1d1-ae75d9da3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.905370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a29bf-f730-481b-b1d1-ae75d9da3ebf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.905827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a29bf-f730-481b-b1d1-ae75d9da3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.906157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a29bf-f730-481b-b1d1-ae75d9da3ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.917243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a29bf-f730-481b-b1d1-ae75d9da3ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:50.918767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a29bf-f730-481b-b1d1-ae75d9da3ebf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:50.933193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8ffa-c11b-4fa2-bb9f-0087a337c37f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7f6a29bf-f730-481b-b1d1-ae75d9da3ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:50.935284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:50.936165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:50.937257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:50.937257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:50.937257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:50.937257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 221/437] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:50.939064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53447-a8c0-4a23-9a23-573d8941494c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.939064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53447-a8c0-4a23-9a23-573d8941494c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.939064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53447-a8c0-4a23-9a23-573d8941494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.957613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c7f53447-a8c0-4a23-9a23-573d8941494c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:50.958930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c7f53447-a8c0-4a23-9a23-573d8941494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:50.973276 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:50.974270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:50.974746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:50.975272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:50.983986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:50.985829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:50.999645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:51.000612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:50:51.001489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:51.004110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 217/438] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:50:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:51.005208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.017716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.031953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.043419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 222/439] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:51.043419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53447-a8c0-4a23-9a23-573d8941494c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.112523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5308b-6a26-4c28-9d29-d402129e1c1f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.153579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53447-a8c0-4a23-9a23-573d8941494c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.156704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.157318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.157598 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.157901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.158546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:51.158846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.160278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.173766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 218/440] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:51.174543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.197992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b0c86-3faa-4cd7-aed2-d12fdc503d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.198761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b0c86-3faa-4cd7-aed2-d12fdc503d69 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.199142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b0c86-3faa-4cd7-aed2-d12fdc503d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.199557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b0c86-3faa-4cd7-aed2-d12fdc503d69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.209893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b0c86-3faa-4cd7-aed2-d12fdc503d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-505b0c86-3faa-4cd7-aed2-d12fdc503d69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:51.214748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:51.215753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:51.216543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 223/441] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:51.217728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691e5cb-5e18-4a1b-a26d-06d239678a8e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.217728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691e5cb-5e18-4a1b-a26d-06d239678a8e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.217999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691e5cb-5e18-4a1b-a26d-06d239678a8e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.223434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 224/442] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:51.223434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691e5cb-5e18-4a1b-a26d-06d239678a8e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.249140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.287401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a41ce-f2c5-4fa1-b59c-c30bf9eda3a8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.288430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.322477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.323078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.323392 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.323665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.324385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:51.324686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.326120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.334548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691e5cb-5e18-4a1b-a26d-06d239678a8e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.343355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 219/443] 10.222.0.36 () {56 vars in 1135 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:51.345335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.368801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0122c-65c7-4516-b687-7e24432d2198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.369422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0122c-65c7-4516-b687-7e24432d2198 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.369694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0122c-65c7-4516-b687-7e24432d2198 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.369960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0122c-65c7-4516-b687-7e24432d2198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.376331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0122c-65c7-4516-b687-7e24432d2198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.377769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0122c-65c7-4516-b687-7e24432d2198 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fbf0122c-65c7-4516-b687-7e24432d2198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:51.388926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:51.389950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:51.390826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 225/444] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:51.391331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4efad-87c3-4a3c-b000-d6cb572b872a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.391331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4efad-87c3-4a3c-b000-d6cb572b872a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.391591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4efad-87c3-4a3c-b000-d6cb572b872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.408952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a3d4efad-87c3-4a3c-b000-d6cb572b872a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:51.410112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a3d4efad-87c3-4a3c-b000-d6cb572b872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:51.444498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520b96b-d21e-4976-abdb-243984317064 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.471044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.474373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 226/445] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:51.475111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4efad-87c3-4a3c-b000-d6cb572b872a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.484621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.485211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.486156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.486156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.486395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:51.486671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.488071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.496949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 220/446] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:51.500114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.535708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.589961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4efad-87c3-4a3c-b000-d6cb572b872a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.605072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3e54b-b6a0-4676-8fdd-703f27fa8d29 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.625583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.626195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.626471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.626738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.637295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:51.639023 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.639664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.639943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.640216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:51.641883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:51.642753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 227/447] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:51.643902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.657423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.659292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:51.674354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:51.675292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:51.676141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:51.676141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:51.676141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:51.676141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 221/448] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:51.676847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.721884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.753911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab044ac5-1acd-4a32-afeb-a406816c91ec None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.763327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.783621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae7a97-9732-4580-9c3e-22ca2ec6a8ad None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.788097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.788695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.788977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.789246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.790491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:51.790794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.792239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.803547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 228/449] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:51.804085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.818461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.819084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.819390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.819659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.829931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.831393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:51.845404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:51.846403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:51.847297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:51.847297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:51.847297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:51.847297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 222/450] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:51.847297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.906484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e1293-9bc6-4c4a-aaf3-8d6471450442 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.935183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.939308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.939909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.940184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.940454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.941102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:51.941399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.942750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.944187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4cf3b-e48f-411a-b423-60dbf81b8196 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:51.951111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 229/451] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:51.951908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:51.976673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:51.977763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:51.978037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:51.978302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:51.978931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:51.980946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:51.982274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:51.982770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:51.993891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 223/452] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:51.994666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.051986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ecd76-04f1-4f9c-ba8a-f4b85c7dcad8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.085877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.086525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.086804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.087073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.091685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b4b59-4b3e-4376-81b7-d76371b5f394 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.096802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:52.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:52.103264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:52.104071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 230/453] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:52.104725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.127494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.128085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.128362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.128632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.129345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] RBAC: Adding query filter params (role.id=f2f75648d0384a4cac1041775239be27, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=74a91af405e24a0b8e9c893e1c299db4) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:52.129650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.131085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.137253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 224/454] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:51 2022] GET /identity/v3/role_assignments?user.id=74a91af405e24a0b8e9c893e1c299db4&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=f2f75648d0384a4cac1041775239be27 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:52.138002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.159534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.216452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.227751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46098b-4534-4309-8874-bbc89f268138 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.252900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf20f9-8ae6-479b-8768-e1e6623b97c9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.265364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.265988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.266303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.266530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.267328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:52.267633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.269175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.286980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 231/455] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:52.286980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.294082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.294848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.295123 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.295440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.306971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.308481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:52.323990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:52.324913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:52.325742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:52.325742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:52.325742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:52.325742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 225/456] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:52.325963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.393986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde55a9-b61d-4afb-aee1-47802ac5da2c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.403806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.433951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.434592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.434870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.435140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.440075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159d524-d979-4d47-ba6b-33e0fb9e782a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.442160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.443637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:52.454663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:52.455780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:52.456815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:52.460494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 232/457] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:52.461111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.475359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.480940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.481543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.481818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.482116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.482771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:52.483072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.484475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.497787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 226/458] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:52.497787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.576340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002a8b0-e9f7-4443-8c3e-9299d2c39d09 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.607547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b852b7-818a-4cf1-abf4-64525b7c6775 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.611037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.611757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.612103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.612442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.621609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.623142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:52.634582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:52.635677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 233/459] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:52.640861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.645851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.653988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.654746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.655025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.655331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.655997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:52.656266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.657651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.674186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 227/460] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:52.674186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.717303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.750829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138a2e3-5981-4951-a689-0dce44c654e0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.778758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3078-d584-4e7c-b33c-c4fe56214790 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.784298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.784968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.785311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.785641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.786374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:52.786747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.788259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.805172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 234/461] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:52.805172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.821889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.822776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.823369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.823646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.834483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.836178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:52.851576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:52.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:52.854499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:52.854499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:52.854499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:52.854499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 228/462] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:52.854499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:52.898714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.909251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df69b2-402f-4596-8481-5e0aec222064 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.944453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:52.951219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:52.951833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:52.952127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:52.952370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:52.953135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:52.953376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:52.954790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:52.959983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6cbb-4642-49d1-9785-7bda235b5eda None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:52.976886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 235/463] 10.222.0.36 () {56 vars in 1137 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:52.976886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.000090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.000716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.001053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.001269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.009554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.011009 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:53.022436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:53.023553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 229/464] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:53.029181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.089456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb9a4-633e-44ed-9832-007092eef177 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.130022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64336b70-ef91-45fd-bb1a-14db7118ee17 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.131713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.131950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:53.132350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.132625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.133523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/5f4baa1481254765b8af7a253806f28f/roles/f2f75648d0384a4cac1041775239be27` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.147844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=5f4baa1481254765b8af7a253806f28f, role_id=f2f75648d0384a4cac1041775239be27)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.149892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.164850 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 236/465] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:52 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/5f4baa1481254765b8af7a253806f28f/roles/f2f75648d0384a4cac1041775239be27 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:53.175397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.184173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.184941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.185279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.185611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.186412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:53.186797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.188676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.202530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 230/466] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:53.203440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.225659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:53.300170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f9d9b-931f-4936-9014-3ab792934d13 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.346575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08874d-4fd5-4c19-8d21-8362b8d9c66d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.347128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.347533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.347878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.348215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.388484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.389484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.389766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.390037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.390854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:53.391075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.393183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.414021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 231/467] 10.222.0.36 () {56 vars in 1137 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:53.415537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.440848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.443443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.445294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:53.492276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:53.529139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaeab6-37d2-4c50-bb60-5a524de76207 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.566855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.567404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.567844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.568181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.573255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 237/468] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:53.574362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.576571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.578095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:53.590238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:53.591151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 232/469] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:53.592591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.683937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:53.700577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4af169-76c9-42da-8e18-52fee213d0ce None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.710130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf57488-1f1e-42de-b524-3064fe9c23a9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.741126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.741833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.742169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.742508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.754017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:53.756904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.757859 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.758884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.759491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.771185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.772824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:53.787580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:53.788510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:53.790044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:53.790044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:53.790044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:53.790044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 233/470] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:53.790879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.828910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.830622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.895485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153df94-35bb-4764-a41f-e1b2e4fafff8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:53.926674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:53.933882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:53.934573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:53.934945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:53.935317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:53.936178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:53.936529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:53.938366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:53.954410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 238/471] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:53.955676 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:53.961152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 234/472] 10.222.0.36 () {56 vars in 1139 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:53.963297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.011287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.068321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4b33-5fc7-4bfe-a2cf-6cbd4c108bd0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.079299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3671e-0f44-4654-a0db-6c35ae58ee77 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.103617 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.104072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.104348 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.104618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.119109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.120713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.121705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.122200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.123292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:54.123750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.125593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.141081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 239/473] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:54.142300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.176590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.189465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.191095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.268022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.272173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07289f36-69a1-4ade-8515-d6cef2dd7825 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.300260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 235/474] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 06:50:54.301393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.313569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.314522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.314945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.315392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.330121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.331924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:54.347704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:54.348636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:54.349474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:54.349474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:54.349474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:54.349474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 240/475] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:54.350610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.396676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.406765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aad2f-52b7-4940-ba6c-698a84cc4c2c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.442338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e010ac-0234-4daf-8603-44f2b6e190e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.442959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e010ac-0234-4daf-8603-44f2b6e190e2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.443248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e010ac-0234-4daf-8603-44f2b6e190e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.443509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e010ac-0234-4daf-8603-44f2b6e190e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.450162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e010ac-0234-4daf-8603-44f2b6e190e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.451636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e010ac-0234-4daf-8603-44f2b6e190e2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.462068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa658e33-0950-429d-97fb-9c44847abea2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-10e010ac-0234-4daf-8603-44f2b6e190e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:54.463888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:54.464705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:54.465552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 236/476] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:54.466417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9b18-6f71-4b85-a15b-7424b5d3bc14 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.466417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9b18-6f71-4b85-a15b-7424b5d3bc14 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.466576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9b18-6f71-4b85-a15b-7424b5d3bc14 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.467938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 237/477] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:54.472363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa9b18-6f71-4b85-a15b-7424b5d3bc14 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.506325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.507308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.507764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.508270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.519456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.521109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:54.535661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:54.536560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:54.537434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:54.537434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:54.537434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:54.538801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 241/478] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:54.540267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.542760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.582374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa9b18-6f71-4b85-a15b-7424b5d3bc14 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.617361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc660c-e71f-48a2-9d8c-cbda3d43df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.618489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc660c-e71f-48a2-9d8c-cbda3d43df15 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.618770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc660c-e71f-48a2-9d8c-cbda3d43df15 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.619037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc660c-e71f-48a2-9d8c-cbda3d43df15 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.619693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc660c-e71f-48a2-9d8c-cbda3d43df15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:54.619993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc660c-e71f-48a2-9d8c-cbda3d43df15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.621350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc660c-e71f-48a2-9d8c-cbda3d43df15 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.632400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 238/479] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:54.633664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6a7e-aa64-439d-b3c8-58639505e95c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.633950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6a7e-aa64-439d-b3c8-58639505e95c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.634217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6a7e-aa64-439d-b3c8-58639505e95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.652309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-719e6a7e-aa64-439d-b3c8-58639505e95c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:54.653498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-719e6a7e-aa64-439d-b3c8-58639505e95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:54.656760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecc55b-c3b3-4790-ad35-95fc0d5c79bb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.657171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.691297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.691960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.692167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.692465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.693096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:54.694025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.694801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.708087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 242/480] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:54.708879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.731112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 239/481] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:54.732184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6a7e-aa64-439d-b3c8-58639505e95c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.794768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.838617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bb3c-283d-4a71-a09d-72a60e67736d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.848877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6a7e-aa64-439d-b3c8-58639505e95c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:54.873359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.873979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.874251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.874513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.875132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:54.875453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.876873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.888320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:54.893120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 243/482] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:54.893120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.893317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:54.893317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:54.893317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:54.904991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:54.906903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:54.909988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:54.921724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:54.922578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:54.923534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:54.926048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 240/483] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:54.927166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:54.994771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734fb9d-1da5-4218-ace9-3a5c39290533 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.017905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:55.031807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.032403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.032913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.033192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.034308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] RBAC: Adding query filter params (role.id=f2f75648d0384a4cac1041775239be27, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=a0a3956bb98749e6bb19ee7f3465eac9) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:55.034873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.036336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.036446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8744e-5283-463f-973f-2d9fe6f1d85c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.051402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 244/484] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/role_assignments?user.id=a0a3956bb98749e6bb19ee7f3465eac9&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=f2f75648d0384a4cac1041775239be27 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:55.051402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:55.086572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.087816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.088217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.088495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.098232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.100313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:55.118308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:55.119243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:55.120195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:55.123362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 241/485] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:55.124284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:55.179594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e802fd0-ff18-46af-b0e2-14a0752427b4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.207656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:55.222986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.223643 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.223908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.224178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.224872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:55.225173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.226622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.241646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 245/486] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:55.242809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:55.259608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bc11a-6784-4a7c-975f-135c31686a5b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.299774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.300630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.301034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.301309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.302486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:55.315717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:55.322341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:55.323337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 242/487] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:55.324784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:55.376182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f4ec3-cacd-4bdb-b3b7-a9a1f25ffdcd None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.416829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.417445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.417726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.417994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.418685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:55.418983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.421074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.437287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 246/488] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:55.438234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:55.447287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6980a-a2ba-4b69-a027-3ca7a11b604c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.468907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:55.484299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.485014 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.485291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.485564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.486277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] RBAC: Adding query filter params (role.id=f2f75648d0384a4cac1041775239be27, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=92c66f0a106f4c8e99ac7075d76767d4) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:55.486752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.488253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.499360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 243/489] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/role_assignments?user.id=92c66f0a106f4c8e99ac7075d76767d4&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=f2f75648d0384a4cac1041775239be27 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:55.499360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:55.574704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b193b-e1cb-4254-bc49-bc7f7dd971ad None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.588512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:55.630260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87af8b-d59f-483d-8cbf-71672e54d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.631733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87af8b-d59f-483d-8cbf-71672e54d726 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.632463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87af8b-d59f-483d-8cbf-71672e54d726 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.632914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87af8b-d59f-483d-8cbf-71672e54d726 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.634084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87af8b-d59f-483d-8cbf-71672e54d726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:55.634942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87af8b-d59f-483d-8cbf-71672e54d726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.637185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87af8b-d59f-483d-8cbf-71672e54d726 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.645401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ac145-1ae3-4783-b2e1-f4ad3742d63e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.655539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 247/490] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:55.656912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ebf4b-9d88-433e-a388-23bca2437ff5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.657198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ebf4b-9d88-433e-a388-23bca2437ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.657472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ebf4b-9d88-433e-a388-23bca2437ff5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.658972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 248/491] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:55.660793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ebf4b-9d88-433e-a388-23bca2437ff5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:55.699667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b3758-05db-4232-8dc4-77800fb9ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.700274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b3758-05db-4232-8dc4-77800fb9ab76 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.700552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b3758-05db-4232-8dc4-77800fb9ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.700835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b3758-05db-4232-8dc4-77800fb9ab76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.701710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b3758-05db-4232-8dc4-77800fb9ab76 None admin] RBAC: Adding query filter params (role.id=f2f75648d0384a4cac1041775239be27, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=56967a5531ea4e308af5b89c13524c14) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:55.701862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b3758-05db-4232-8dc4-77800fb9ab76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.704060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b3758-05db-4232-8dc4-77800fb9ab76 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.710102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 244/492] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/role_assignments?user.id=56967a5531ea4e308af5b89c13524c14&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=f2f75648d0384a4cac1041775239be27 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:55.712700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07322e83-f9aa-4d93-854a-ce25d4a72da5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.712700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07322e83-f9aa-4d93-854a-ce25d4a72da5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.712700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07322e83-f9aa-4d93-854a-ce25d4a72da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.739237 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-07322e83-f9aa-4d93-854a-ce25d4a72da5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:55.740139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-07322e83-f9aa-4d93-854a-ce25d4a72da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:55.798308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:55.807646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ebf4b-9d88-433e-a388-23bca2437ff5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:55.851618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:55.853096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:55.853991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:55.854588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:55.865548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 245/493] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:55.871460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:55.873363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:55.895442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:55.896362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:55.897807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:55.897807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:55.897807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:55.900059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 249/494] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:55.905451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07322e83-f9aa-4d93-854a-ce25d4a72da5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.032340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07322e83-f9aa-4d93-854a-ce25d4a72da5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:56.073202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:56.073818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:56.074096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:56.074369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:56.075021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:56.075354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:56.076766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:56.092887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 246/495] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:56.098027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.133910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:56.200085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.230612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec5778-9df8-4970-bc45-2441461473cf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:56.262341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:56.285469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:56.286229 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:56.286508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:56.286778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:56.300258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:56.301990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:56.324310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:56.325276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:50:56.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:56.328806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 250/496] 10.222.0.36 () {56 vars in 1093 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:56.331251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cb808-62a5-43e5-b060-2c03c69ba5ac None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:56.333989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.381481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:56.382104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:56.382375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:56.382641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:56.410865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:50:56.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:56.420465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:56.421290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 247/497] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:56.426696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.488396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:56.510295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd50cd0-2022-43ce-a6d6-5e45f8bdc359 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:56.546557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:56.547555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:56.548038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:56.548451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:56.549528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:56.550035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:56.551754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:56.572599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 251/498] 10.222.0.36 () {56 vars in 1133 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:50:56.578355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.624075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:56.630429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8434930-88e2-41e3-9f72-a7bc339d43e6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:56.702567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:56.703423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:56.703868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:56.704285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:56.706868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:56.707382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:56.708906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:56.729180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 248/499] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:56.734859 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.777935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ebb65-3ec5-4777-87ba-942827594848 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:56.853750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:56.854390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:56.854672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:56.854947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:56.863660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:56.874862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:56.876490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:56.879010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682aac93-6aee-40cb-835f-39c8c4a6c25c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:56.906986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:56.908049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:56.909435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:56.909435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:56.909435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:56.912039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 252/500] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:56.919286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:56.934957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:56.935926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:56.936404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:56.937122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:56.949539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:56.951428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:56.967795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:56.968718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:56.969610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:56.969610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:56.969610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:56.971831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 249/501] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:56.986352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:57.018616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:57.116961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6884-347b-49b2-8d2c-a75772d5d7f9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:57.145600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f967e49-c7f2-42cd-ad42-2a5182aeef90 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:57.180800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:57.181411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.181690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.181963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.182593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:57.182896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:57.184579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:57.191909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:57.192530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.192812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.193086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.193729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:57.194027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:57.195420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:57.199187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 250/502] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:57.208934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:57.213137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 253/503] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:57.218630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:57.325244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:57.355654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:57.374574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39789f3f-9f18-4331-8c38-bd05ae2d6d8a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:57.413491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:57.414818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.415260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.415751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.424093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:57.425652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:57.425819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053ae30-c004-4692-bd11-7cfc4f17c89d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:50:57.439777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:57.440736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:50:57.441769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:57.446714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 251/504] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:50:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:57.448586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:57.508586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:57.509144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.509423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.509696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.520508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:57.521964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:57.545868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:57.546815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:57.547752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 254/505] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:50:57.552712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:57.612925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811f026-d976-4826-aaa9-bfcd0e183b84 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:57.651431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:57.659851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e88c52-82b1-422f-a60b-cefdbac6a552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:57.661045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88c52-82b1-422f-a60b-cefdbac6a552 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.661531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88c52-82b1-422f-a60b-cefdbac6a552 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.662228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88c52-82b1-422f-a60b-cefdbac6a552 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.663569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88c52-82b1-422f-a60b-cefdbac6a552 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:57.664084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88c52-82b1-422f-a60b-cefdbac6a552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:57.666436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88c52-82b1-422f-a60b-cefdbac6a552 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:57.680797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734e119-f0fe-4e6a-9771-31c10a3a8fa5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:57.692431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:57.692737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 252/506] 10.222.0.36 () {56 vars in 1135 bytes} [Mon Oct 17 06:50:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:57.695130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6fbd0-8860-40f5-8f29-ee323967a934 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.695911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6fbd0-8860-40f5-8f29-ee323967a934 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.696419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6fbd0-8860-40f5-8f29-ee323967a934 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.698731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 253/507] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:57.699765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6fbd0-8860-40f5-8f29-ee323967a934 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:57.722761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46028a6-5c08-4dd3-ac2f-9f4def4c92f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:57.723390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46028a6-5c08-4dd3-ac2f-9f4def4c92f5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.723663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46028a6-5c08-4dd3-ac2f-9f4def4c92f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.723929 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46028a6-5c08-4dd3-ac2f-9f4def4c92f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.724597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46028a6-5c08-4dd3-ac2f-9f4def4c92f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:57.724903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46028a6-5c08-4dd3-ac2f-9f4def4c92f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:57.726312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46028a6-5c08-4dd3-ac2f-9f4def4c92f5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:57.738714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 255/508] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:57.740050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614c61-ac79-4b07-9ead-cee14f8db31b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.740327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614c61-ac79-4b07-9ead-cee14f8db31b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.740594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614c61-ac79-4b07-9ead-cee14f8db31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:57.760024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0f614c61-ac79-4b07-9ead-cee14f8db31b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:57.761118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0f614c61-ac79-4b07-9ead-cee14f8db31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:57.833373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 256/509] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:57.839918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614c61-ac79-4b07-9ead-cee14f8db31b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:57.840768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6fbd0-8860-40f5-8f29-ee323967a934 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:57.917622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f1a6c-ce91-482d-a668-2b217aba3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:57.918625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f1a6c-ce91-482d-a668-2b217aba3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:57.919117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f1a6c-ce91-482d-a668-2b217aba3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:57.919621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f1a6c-ce91-482d-a668-2b217aba3aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.033952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.037996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614c61-ac79-4b07-9ead-cee14f8db31b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:58.063304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.081138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:58.082118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.082840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.083347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/74a91af405e24a0b8e9c893e1c299db4/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.085469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=74a91af405e24a0b8e9c893e1c299db4, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.087142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.093435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f1a6c-ce91-482d-a668-2b217aba3aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.095172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f1a6c-ce91-482d-a668-2b217aba3aa3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.104466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 257/510] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:50:57 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/74a91af405e24a0b8e9c893e1c299db4/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:50:58.114415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:58.265812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3862c97-ff87-4200-96b4-6f4a60758311 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:58.278297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 254/511] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:50:58.279626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08247d-fd1b-4b6d-85dc-7d6d0f39ab03 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.279947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08247d-fd1b-4b6d-85dc-7d6d0f39ab03 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.280219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08247d-fd1b-4b6d-85dc-7d6d0f39ab03 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.285895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 255/512] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:58.287110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08247d-fd1b-4b6d-85dc-7d6d0f39ab03 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:58.310797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea457ac-b3d0-4972-83f3-a6dd1e0e60f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:58.311447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea457ac-b3d0-4972-83f3-a6dd1e0e60f4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.311741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea457ac-b3d0-4972-83f3-a6dd1e0e60f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.312277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea457ac-b3d0-4972-83f3-a6dd1e0e60f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.322743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea457ac-b3d0-4972-83f3-a6dd1e0e60f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.324226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea457ac-b3d0-4972-83f3-a6dd1e0e60f4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2ea457ac-b3d0-4972-83f3-a6dd1e0e60f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:58.341549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:58.343149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:58.344098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:58.344098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:58.344098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:58.344098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 258/513] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:58.344320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bc3f1-8f75-44e8-abc0-5e511bfedf12 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.344591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bc3f1-8f75-44e8-abc0-5e511bfedf12 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.344868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bc3f1-8f75-44e8-abc0-5e511bfedf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.362544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-313bc3f1-8f75-44e8-abc0-5e511bfedf12 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:58.363749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-313bc3f1-8f75-44e8-abc0-5e511bfedf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:58.371639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.393841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.406487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08247d-fd1b-4b6d-85dc-7d6d0f39ab03 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:58.437540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 259/514] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:58.439075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea1fc-8395-4700-bd0d-bf5c423580f9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.439392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea1fc-8395-4700-bd0d-bf5c423580f9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.439664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea1fc-8395-4700-bd0d-bf5c423580f9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.445430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 260/515] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:50:58.445430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ea1fc-8395-4700-bd0d-bf5c423580f9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:58.453316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8fdf3-aad6-4da4-aa06-6caf4fe1e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:58.454287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8fdf3-aad6-4da4-aa06-6caf4fe1e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.454790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8fdf3-aad6-4da4-aa06-6caf4fe1e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.455305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8fdf3-aad6-4da4-aa06-6caf4fe1e2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.466435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8fdf3-aad6-4da4-aa06-6caf4fe1e2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.468186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8fdf3-aad6-4da4-aa06-6caf4fe1e2e0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a5b8fdf3-aad6-4da4-aa06-6caf4fe1e2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:50:58.484218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:58.485181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:50:58.486162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:50:58.486162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:50:58.486162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:58.487950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 256/516] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:58.494318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd855-2f90-4926-967f-7f224b72c4a8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.494821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd855-2f90-4926-967f-7f224b72c4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.495566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd855-2f90-4926-967f-7f224b72c4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.515931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b0cfd855-2f90-4926-967f-7f224b72c4a8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:50:58.517236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b0cfd855-2f90-4926-967f-7f224b72c4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:50:58.561242 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ea1fc-8395-4700-bd0d-bf5c423580f9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:58.595509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 257/517] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:50:58.596397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfd855-2f90-4926-967f-7f224b72c4a8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:58.611902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:58.612827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.613309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.613779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.614642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:58.615144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.616997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.641764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 261/518] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:58.647613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:58.652100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.720101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.757333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfd855-2f90-4926-967f-7f224b72c4a8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:58.792782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:58.793720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.793720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.794013 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.794622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:58.794922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.796329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.813339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 258/519] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:58.813339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:58.823145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997396a-dbea-4f05-bb9a-b5b336c941f4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:58.864766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:58.865615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.866441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.866898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.868079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.869655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.898997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 262/520] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:50:58 2022] POST /identity/v3/users => generated 254 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:50:58.900473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:58.917219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2bfd-07df-4e8d-afc9-7b3556597c3a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:58.952616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.958910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:58.960063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:58.960440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:58.960802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:58.961650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:58.963156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:58.974669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:58.977560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:58.978603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:50:58.979484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:58.985560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 259/521] 10.222.0.36 () {56 vars in 1129 bytes} [Mon Oct 17 06:50:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:58.986622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:59.035376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc5ef4-9234-4273-af36-defba8bb7e3a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.073124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.073816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.074159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.074498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.084328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.085924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:50:59.102444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:59.103441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:50:59.104768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:50:59.107448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 263/522] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:59.108161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:59.125417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136050f1-877a-4092-8aec-062be93ecff2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.177823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.178485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.178764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.179039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.180295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.181796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:50:59.191543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:50:59.192533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:50:59.193388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 260/523] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:50:58 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:50:59.194171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:59.236717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e450939-f4fa-4907-858b-5f787d59fa61 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.260100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:59.263257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:59.285205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.286534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.287458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.287920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.288992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:59.289553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.291867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.298305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 264/524] 10.222.0.36 () {56 vars in 1124 bytes} [Mon Oct 17 06:50:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:59.299654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:59.311366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e954-b005-46ea-afb4-825398ad55c5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.352455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.352985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.353264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.353536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.354220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:59.354521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.356003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.373687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 261/525] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:59.374783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:59.430768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9f45c-f454-44cd-9e58-d8a486551fd0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.474731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.476026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.476519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.477809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.478903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] RBAC: Adding query filter params (name=network) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:59.479411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.481538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.487925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 265/526] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:50:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:59.488680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:59.509589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccef31-766b-4fb4-8bb6-5f01ccee2d9b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.537223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:59.538149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:50:59.559535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.560499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.560980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.561692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.563894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:59.564416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.566473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.577218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 262/527] 10.222.0.36 () {56 vars in 1124 bytes} [Mon Oct 17 06:50:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:59.578744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:50:59.650076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49337a-2174-4f32-ac76-4fd1bc9ce956 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.704754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e6665-5729-458b-a954-25f3fe77ab79 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:50:59.744452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e0ef-6e47-477b-830a-8ec4fb03e055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.745894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e0ef-6e47-477b-830a-8ec4fb03e055 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.746797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e0ef-6e47-477b-830a-8ec4fb03e055 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.747629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e0ef-6e47-477b-830a-8ec4fb03e055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.749084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59e0ef-6e47-477b-830a-8ec4fb03e055 None admin] RBAC: Adding query filter params (role.id=f2f75648d0384a4cac1041775239be27, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=5f4baa1481254765b8af7a253806f28f) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:59.749915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68dc1b-f9c1-403a-b36b-1f9b57b5bc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:50:59.751087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68dc1b-f9c1-403a-b36b-1f9b57b5bc48 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:50:59.751691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59e0ef-6e47-477b-830a-8ec4fb03e055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.753330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68dc1b-f9c1-403a-b36b-1f9b57b5bc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:50:59.753815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68dc1b-f9c1-403a-b36b-1f9b57b5bc48 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:50:59.754675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68dc1b-f9c1-403a-b36b-1f9b57b5bc48 None admin] RBAC: Adding query filter params (type=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:50:59.755388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59e0ef-6e47-477b-830a-8ec4fb03e055 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.755923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68dc1b-f9c1-403a-b36b-1f9b57b5bc48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:50:59.759193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68dc1b-f9c1-403a-b36b-1f9b57b5bc48 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:50:59.771173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 263/528] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:50:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:50:59.771972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 266/529] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:50:59 2022] GET /identity/v3/role_assignments?user.id=5f4baa1481254765b8af7a253806f28f&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=f2f75648d0384a4cac1041775239be27 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:00.345537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f7b13-9b88-4a74-8178-458a065f0cbf None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:00.345926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f7b13-9b88-4a74-8178-458a065f0cbf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:00.346108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f7b13-9b88-4a74-8178-458a065f0cbf None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:00.347699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 267/530] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:00.354476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c18467-b7da-4340-9341-7695e20a0130 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:00.354764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c18467-b7da-4340-9341-7695e20a0130 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:00.355042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c18467-b7da-4340-9341-7695e20a0130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:00.377938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f4c18467-b7da-4340-9341-7695e20a0130 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:00.379124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f4c18467-b7da-4340-9341-7695e20a0130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:00.437326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:00.472845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 264/531] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:51:00.764928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f7b13-9b88-4a74-8178-458a065f0cbf None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:00.895947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f7b13-9b88-4a74-8178-458a065f0cbf None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:00.912701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:00.945207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:00.946432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:00.946711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:00.946982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:00.961525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:00.966165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:00.967091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:00.967980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 268/532] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:51:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:00.975110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c18467-b7da-4340-9341-7695e20a0130 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:01.117701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c18467-b7da-4340-9341-7695e20a0130 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:01.163148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:01.163788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:01.164591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:01.164862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:01.165523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:01.165825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:01.167194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:01.180994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 265/533] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:01.186602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:01.347128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a324ee-6cd0-4dab-abaa-a32ff458ada9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:01.395992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86217e6b-8646-4cda-8402-69b4e5776815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:01.396551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86217e6b-8646-4cda-8402-69b4e5776815 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:01.396833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86217e6b-8646-4cda-8402-69b4e5776815 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:01.397106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86217e6b-8646-4cda-8402-69b4e5776815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:01.410711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86217e6b-8646-4cda-8402-69b4e5776815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:01.412191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86217e6b-8646-4cda-8402-69b4e5776815 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-86217e6b-8646-4cda-8402-69b4e5776815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:01.429487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:01.430966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:01.431897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:01.431897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:01.431897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:01.431897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 269/534] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:01.440276 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:01.513547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19ca4-704b-4324-a8a3-15fd429e1f82 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:01.513857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19ca4-704b-4324-a8a3-15fd429e1f82 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:01.514098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19ca4-704b-4324-a8a3-15fd429e1f82 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:01.515463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 270/535] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:01.515889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:01.528463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700fb6-1059-4d7c-aa91-b9216917af76 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:01.528755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700fb6-1059-4d7c-aa91-b9216917af76 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:01.529021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700fb6-1059-4d7c-aa91-b9216917af76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:01.569133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cf700fb6-1059-4d7c-aa91-b9216917af76 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:01.570386 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cf700fb6-1059-4d7c-aa91-b9216917af76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:01.603550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:01.619557 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17132561-fdcb-449d-b75b-02debb390e9d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:01.660537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:01.661643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:01.662849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:01.663572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:01.665359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:01.666473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:01.668179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:01.691887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 266/536] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:01.702146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:01.709610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 271/537] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:51:01.821639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:01.825516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c00a-aa3b-44de-9918-4652aa333951 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:01.868983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:01.869952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:01.870455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:01.870924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:01.878858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:01.880566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:01.892399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:01.893342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:51:01.894274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:01.896781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 267/538] 10.222.0.36 () {56 vars in 1093 bytes} [Mon Oct 17 06:51:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:01.902693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf700fb6-1059-4d7c-aa91-b9216917af76 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:02.007129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf700fb6-1059-4d7c-aa91-b9216917af76 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:02.043099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:02.044113 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.044601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.045070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.045999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:02.046506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:02.048051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:02.058787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:02.094950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 272/539] 10.222.0.36 () {56 vars in 1133 bytes} [Mon Oct 17 06:51:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:02.101304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:02.121990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:02.228175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185687cc-4892-421d-8888-161f3573e6f6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:02.301999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:02.307936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.308232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.308493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.309688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd21d7-1087-4a6e-99cd-6a071e72875c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:02.330239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:02.335484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:02.336948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:51:02.337780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:02.343755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 268/540] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:51:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:02.350099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:02.353941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095856b9-32e5-4c82-a660-434a73ac59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:02.354987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095856b9-32e5-4c82-a660-434a73ac59d5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.355540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095856b9-32e5-4c82-a660-434a73ac59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.356022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095856b9-32e5-4c82-a660-434a73ac59d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.385105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:02.459431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095856b9-32e5-4c82-a660-434a73ac59d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:02.460956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095856b9-32e5-4c82-a660-434a73ac59d5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:02.472719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:02.496131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69982-8ac0-4b5e-bf44-d5908a72a323 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:02.530692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e61e09-8088-451c-8700-fa30d2f60a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:02.531325 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61e09-8088-451c-8700-fa30d2f60a72 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.531606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61e09-8088-451c-8700-fa30d2f60a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.531877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61e09-8088-451c-8700-fa30d2f60a72 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.532533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e61e09-8088-451c-8700-fa30d2f60a72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:02.532838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e61e09-8088-451c-8700-fa30d2f60a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:02.534202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e61e09-8088-451c-8700-fa30d2f60a72 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:02.546494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 269/541] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:51:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:02.546494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526f377-4611-47d7-b65c-2d4b3f45bd51 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.546494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526f377-4611-47d7-b65c-2d4b3f45bd51 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.546494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526f377-4611-47d7-b65c-2d4b3f45bd51 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.546494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 270/542] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:02.548334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e955b47-75b9-4081-8ada-d52f0f95fd72 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.548613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e955b47-75b9-4081-8ada-d52f0f95fd72 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.548886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e955b47-75b9-4081-8ada-d52f0f95fd72 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.550805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 271/543] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:02.551520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e955b47-75b9-4081-8ada-d52f0f95fd72 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:02.575373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 273/544] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:51:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:51:02.576973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4258d-5d45-4030-b164-6d5eb04eea2e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.577461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4258d-5d45-4030-b164-6d5eb04eea2e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.577917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4258d-5d45-4030-b164-6d5eb04eea2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.597347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d3b4258d-5d45-4030-b164-6d5eb04eea2e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:02.598580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d3b4258d-5d45-4030-b164-6d5eb04eea2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:02.646311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:02.667617 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e955b47-75b9-4081-8ada-d52f0f95fd72 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:02.673348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 274/545] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:51:02.674576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6da3a9-4ef2-4de0-9e81-73a5bfa38074 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.674854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6da3a9-4ef2-4de0-9e81-73a5bfa38074 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.675121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6da3a9-4ef2-4de0-9e81-73a5bfa38074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.693351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bd6da3a9-4ef2-4de0-9e81-73a5bfa38074 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:02.694523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bd6da3a9-4ef2-4de0-9e81-73a5bfa38074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:02.715294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:02.716004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.716283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.716830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.727997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:02.729477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:02.732065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:02.744760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:02.745731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:02.746642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:02.746642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:02.746642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:02.746642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 272/546] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:02.746642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:02.775325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 275/547] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:51:02.776843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d9338-a539-4c7d-ba22-d3123e591be3 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.777124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d9338-a539-4c7d-ba22-d3123e591be3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.777394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d9338-a539-4c7d-ba22-d3123e591be3 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.778747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 276/548] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:02.779955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d9338-a539-4c7d-ba22-d3123e591be3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:02.870243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2f634-86e7-4cd9-874f-08d517e33151 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:02.905324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfd7e7-6e6a-441a-b19f-a668dec3cecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:02.906119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfd7e7-6e6a-441a-b19f-a668dec3cecf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.906402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfd7e7-6e6a-441a-b19f-a668dec3cecf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.906673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfd7e7-6e6a-441a-b19f-a668dec3cecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.907824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d9338-a539-4c7d-ba22-d3123e591be3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:02.908720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:02.918811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfd7e7-6e6a-441a-b19f-a668dec3cecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:02.920421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfd7e7-6e6a-441a-b19f-a668dec3cecf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c1cfd7e7-6e6a-441a-b19f-a668dec3cecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:02.935950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:02.937217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:02.938107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:02.938107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:02.938107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:02.938107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 273/549] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:02.938935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3a922-7cc5-41ca-bc8a-b98c775e1aac None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.939247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3a922-7cc5-41ca-bc8a-b98c775e1aac None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.939551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3a922-7cc5-41ca-bc8a-b98c775e1aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.957787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:02.958235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-51d3a922-7cc5-41ca-bc8a-b98c775e1aac None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:02.958947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:02.959490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-51d3a922-7cc5-41ca-bc8a-b98c775e1aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:02.959832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:02.960298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:02.961660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:02.962430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:02.964668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:02.980432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 277/550] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:02.980432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.007705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:03.036777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 274/551] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:51:03.037626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3a922-7cc5-41ca-bc8a-b98c775e1aac None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.096506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b24283-fd2d-425e-8ac2-b71043686465 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.131822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.132506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.132847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.133121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.133914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:03.134548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.136276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.142912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3a922-7cc5-41ca-bc8a-b98c775e1aac None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.157197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 278/552] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:03.158140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.166433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:03.186679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.187578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.187852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.188121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.188752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.190116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.209989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 275/553] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:03 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:03.210580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.277777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:03.292402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648371cd-5fde-40ae-b252-6dae02b8ece8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.334254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ea3f6-275f-472a-a369-413112206b3e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.347358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.348551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.348841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.349116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.358657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.360221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:03.373597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:03.374649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 279/554] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:03.376127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.384059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.384636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.384916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.385191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.385854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.387324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.407407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 276/555] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:03 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:03.411260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.453787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:03.507376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af95bc-5368-4e0b-8a7e-027065f43557 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.548675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd130a-1b30-470e-851a-185b35b41173 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.550222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.550893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.551256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.551422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.559735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.561235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:03.574340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:03.575346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 280/556] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:03.576623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.579768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:03.597917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.598905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.599183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.599482 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.611453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:03.616197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:03.617142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:03.618007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 277/557] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:03.618738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.697403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ed49-3f7a-45fe-a85d-e87bd33d9c8b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.731415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:03.741017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.741734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.742606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.742943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.743763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:03.744148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.745655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.749832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864a7a-cfd9-41b6-8222-99dfb84c7763 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.770057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 281/558] 10.222.0.36 () {56 vars in 1149 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:03.770057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.792741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.793784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.794558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.795118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.796105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:03.796765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.798981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.815590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 278/559] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:03.816846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.836746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:03.881798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbab23-5ace-4f42-8405-f194351dd4cf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.919652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.920301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.920624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.920973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.921693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:03.922062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.923576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:03.937657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 282/560] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:03.937657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:03.942783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63157577-3437-4798-bb00-6b65d174c8bf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:03.985751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:03.986306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:03.986584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:03.986857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:03.997608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:03.999059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.008151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:04.017318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:04.018377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:04.019365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:04.019365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:04.019365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:04.020708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 279/561] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:04.020708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:04.049531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7e2f-f73f-4e6e-8ec7-35a0c82dc0d5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:04.074894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:04.088475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:04.089374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:04.089766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:04.090447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:04.091550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=74a91af405e24a0b8e9c893e1c299db4) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:04.091960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:04.093473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.100349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 283/562] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:51:03 2022] GET /identity/v3/role_assignments?user.id=74a91af405e24a0b8e9c893e1c299db4&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:04.103184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:04.175403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b763ba-70b6-4729-ae8a-8ef9eeefadda None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:04.250366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d564873-c60e-465c-872c-f199843be53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:04.253030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564873-c60e-465c-872c-f199843be53c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:04.253462 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564873-c60e-465c-872c-f199843be53c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:04.255437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564873-c60e-465c-872c-f199843be53c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:04.292194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d564873-c60e-465c-872c-f199843be53c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:04.293647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d564873-c60e-465c-872c-f199843be53c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.303828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0a529-d4ea-4f2c-a5d8-198b904552bc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7d564873-c60e-465c-872c-f199843be53c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:04.323235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:04.324239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:04.325112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:04.325112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:04.325112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:04.325112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 280/563] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:04.332151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d564873-c60e-465c-872c-f199843be53c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:04.350345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:04.350959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:04.351250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:04.352083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:04.352718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:04.353018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:04.354384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.368041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 284/564] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:04.373409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:04.401953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:04.434461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:04.475753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d564873-c60e-465c-872c-f199843be53c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:04.501197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241196a0-60e1-4135-81b2-2abc654f325d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:04.515760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:04.516458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:04.516748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:04.517026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:04.517766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:04.518074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:04.519517 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.534791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 281/565] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:04.540547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:04.543459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:04.544589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:04.545927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:04.546344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:04.555591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:04.557459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:04.582390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:04.583551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:04.584644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:04.588455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 285/566] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:04.600165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:04.693673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30c96-7ea5-458a-a82c-cf5f415c60a9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:04.715905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:04.760547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:04.761107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:04.761379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:04.761645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:04.766315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d825d0d-2cef-43b7-b72c-83f0b2e32f31 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:04.774407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:04.783985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:04.785470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:04.800228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:04.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:04.802589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 282/567] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:51:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:04.810923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:04.830208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:04.830810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:04.831085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:04.831388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:04.832068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:04.832369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:04.833813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:04.849312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 286/568] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:04.855677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:05.001176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f605-f120-4130-a1b9-e5a10291b741 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:05.030628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6b217-32b7-4ff5-858a-ae25e74af764 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:05.088147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:05.088814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:05.089093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:05.089367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/96d71699c4c648529f5c6a9dd1ef6ca3/roles/f2f75648d0384a4cac1041775239be27` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:05.123440 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=96d71699c4c648529f5c6a9dd1ef6ca3, role_id=f2f75648d0384a4cac1041775239be27)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:05.124895 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:05.129840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:05.135232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:05.135232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:05.135232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:05.135232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:05.135232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:05.135232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:05.163254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 287/569] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:51:04 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/96d71699c4c648529f5c6a9dd1ef6ca3/roles/f2f75648d0384a4cac1041775239be27 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:51:05.168040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 283/570] 10.222.0.36 () {56 vars in 1135 bytes} [Mon Oct 17 06:51:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:05.187238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:05.227650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:05.359898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3e2e4-1b5d-43b6-a332-8eed3a7e66df None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:05.437529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:05.438219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:05.438493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:05.438756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:05.454309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:05.455881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:05.488010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:05.489095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:05.489981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:05.489981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:05.489981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:05.489981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 288/571] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:05.495109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:05.552502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:05.643921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d237065-0e22-413b-b05f-343deccb37d0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:05.680652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac61a70-851c-4a0f-9ee6-6276f0578644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:05.681251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac61a70-851c-4a0f-9ee6-6276f0578644 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:05.681526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac61a70-851c-4a0f-9ee6-6276f0578644 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:05.681794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac61a70-851c-4a0f-9ee6-6276f0578644 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:05.682428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac61a70-851c-4a0f-9ee6-6276f0578644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:05.682728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac61a70-851c-4a0f-9ee6-6276f0578644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:05.684147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac61a70-851c-4a0f-9ee6-6276f0578644 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:05.697062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 284/572] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:05.704300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:05.797731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57739c0a-2801-4e40-b25f-d61ee429479c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:05.798082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57739c0a-2801-4e40-b25f-d61ee429479c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:05.798274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57739c0a-2801-4e40-b25f-d61ee429479c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:05.800529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 285/573] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:05.814660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55b324-a05b-46ed-84f4-16d0d8159af4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:05.814957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55b324-a05b-46ed-84f4-16d0d8159af4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:05.815289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55b324-a05b-46ed-84f4-16d0d8159af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:05.823565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:05.855561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1f55b324-a05b-46ed-84f4-16d0d8159af4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:05.856385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cec97-be06-4a91-b035-c722a20067b5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:05.856769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1f55b324-a05b-46ed-84f4-16d0d8159af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:05.891389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:05.892044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:05.892428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:05.892700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:05.900764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:05.902241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:05.913718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:05.914642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:05.915617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:05.919337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 289/574] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:05.920692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:05.997185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 286/575] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:06.027380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504afdf1-8b1e-42d5-bc06-fc3d4844bd92 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:06.056145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:06.062486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fbd48-1125-470d-ba37-33bd8c350f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:06.063321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fbd48-1125-470d-ba37-33bd8c350f38 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.063765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fbd48-1125-470d-ba37-33bd8c350f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.064158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fbd48-1125-470d-ba37-33bd8c350f38 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.064904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fbd48-1125-470d-ba37-33bd8c350f38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:06.065388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fbd48-1125-470d-ba37-33bd8c350f38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:06.066813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fbd48-1125-470d-ba37-33bd8c350f38 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:06.083380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 290/576] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:06.085250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55b324-a05b-46ed-84f4-16d0d8159af4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:06.093628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07567a-db52-4a21-b767-e728d5cdc89b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.093902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07567a-db52-4a21-b767-e728d5cdc89b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.094171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07567a-db52-4a21-b767-e728d5cdc89b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.095765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 291/577] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:06.112535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d376e-0dc0-47fe-b107-40da1411fc39 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.112834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d376e-0dc0-47fe-b107-40da1411fc39 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.113062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d376e-0dc0-47fe-b107-40da1411fc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.128078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:06.140348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-994d376e-0dc0-47fe-b107-40da1411fc39 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:06.142257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-994d376e-0dc0-47fe-b107-40da1411fc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:06.272065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55b324-a05b-46ed-84f4-16d0d8159af4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:06.276324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 292/578] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:06.308594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:06.309250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.309572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.309843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/5f4baa1481254765b8af7a253806f28f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.312080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=5f4baa1481254765b8af7a253806f28f, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:06.313587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:06.335489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 287/579] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:51:06 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/5f4baa1481254765b8af7a253806f28f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:51:06.384599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d376e-0dc0-47fe-b107-40da1411fc39 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:06.520297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d376e-0dc0-47fe-b107-40da1411fc39 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:06.547615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:06.568983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:06.569609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.569881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.570208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] PATH_INFO: `/v3/services/network` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.570787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:06.572184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:06.583056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:06.585405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:06.587098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 293/580] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:06 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:06.596451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:06.623383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:06.723509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:06.760351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c76e81-8fe1-41b5-9e4f-284324e6e2fb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:06.769591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f7c52-4deb-48cc-927f-ea781a79cc36 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:06.809154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc170b08-5f22-4ea9-a35f-a9879dcc04de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:06.809747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc170b08-5f22-4ea9-a35f-a9879dcc04de None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.809962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc170b08-5f22-4ea9-a35f-a9879dcc04de None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.810231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc170b08-5f22-4ea9-a35f-a9879dcc04de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.810805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc170b08-5f22-4ea9-a35f-a9879dcc04de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:06.812472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc170b08-5f22-4ea9-a35f-a9879dcc04de None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:06.814292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:06.815476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.815955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.816816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.818163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] RBAC: Adding query filter params (name=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:06.818891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:06.820718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:06.826613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 288/581] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-cc170b08-5f22-4ea9-a35f-a9879dcc04de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:06.830912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:06.831978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:06.832856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 294/582] 10.222.0.36 () {56 vars in 1129 bytes} [Mon Oct 17 06:51:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:06.833663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:06.833734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ee62b-fde2-45f8-ac90-764d04b56f22 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.833734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ee62b-fde2-45f8-ac90-764d04b56f22 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.833734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ee62b-fde2-45f8-ac90-764d04b56f22 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.836639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 295/583] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:06.841212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce7b0b-0de9-4578-8abb-683e9e144ab3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:06.841475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce7b0b-0de9-4578-8abb-683e9e144ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:06.841755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce7b0b-0de9-4578-8abb-683e9e144ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:06.860000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c1ce7b0b-0de9-4578-8abb-683e9e144ab3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:06.861170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c1ce7b0b-0de9-4578-8abb-683e9e144ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:06.906680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:06.939340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 296/584] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:06.940247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce7b0b-0de9-4578-8abb-683e9e144ab3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:06.969351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4f32b-aba0-42cc-94d9-6199b40fed3d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.008909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.009903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.010382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.010867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.011724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:07.012668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.014246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.021246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 289/585] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:07.028113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.043143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:07.123135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce7b0b-0de9-4578-8abb-683e9e144ab3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.159764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.160668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.161258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.161375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.162066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:07.162458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.167230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.173739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 297/586] 10.222.0.36 () {56 vars in 1124 bytes} [Mon Oct 17 06:51:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:07.174994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.190932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beb504-900c-489d-aa8d-5e71cf5d1ed8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.217542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:07.245343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.246101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.251041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.251041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.251041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] RBAC: Adding query filter params (interface=public, service_id=a40e352271d342e5bfb737fb6f61d27e, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:07.251041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.251041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.259605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 290/587] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/endpoints?service_id=a40e352271d342e5bfb737fb6f61d27e&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:07.265174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.305295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62f52c-8d4e-4844-a47d-5d2945591d06 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.342511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.343107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.344061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.344310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.344932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:07.345232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.346576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.355257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 298/588] 10.222.0.36 () {56 vars in 1124 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:07.355257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.360460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:07.392303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257d250-b5ce-4027-b4ee-d29a24288a0b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.428160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.428788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.429033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.429848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.430661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.432098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.437916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 291/589] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:07.438763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.483280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e5bb0-0d64-4976-a7c1-e2b770c114cf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.487029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:07.522511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.524084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.524864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.525353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.526160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.527952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:07.537683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:07.538788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:07.539687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:07.542800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 299/590] 10.222.0.36 () {56 vars in 1132 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:07.553115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.665800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4385aa-b454-4293-ac13-b5ed02059464 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.705085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f95786-bf76-427e-89f0-33c6c4e06adb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.722187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.723318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.724207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.724651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.725750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] RBAC: Adding query filter params (interface=public, service_id=3f21f1d4f345411f8e68c92320ae5efa, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:07.726734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.729025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.732662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:07.743190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 292/591] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/endpoints?service_id=3f21f1d4f345411f8e68c92320ae5efa&interface=public®ion_id=RegionOne => generated 194 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:07.744692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.754281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.754892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.755297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.755563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.756238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:07.756535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.759418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.765462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 300/592] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:07.770869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:07.810211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:07.883460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3782-f386-4824-9f96-9ce8cf45a52d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.913438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ab52e-a7cd-4837-a7a8-2bcea71d4501 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:07.924817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf174d8-4420-4d72-a7b4-1280188988f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.925358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf174d8-4420-4d72-a7b4-1280188988f4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.925623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf174d8-4420-4d72-a7b4-1280188988f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.925895 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf174d8-4420-4d72-a7b4-1280188988f4 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.926516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf174d8-4420-4d72-a7b4-1280188988f4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:07.926813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf174d8-4420-4d72-a7b4-1280188988f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.928208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf174d8-4420-4d72-a7b4-1280188988f4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.933633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 301/593] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:07.934864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e6e28-e818-4581-b1fc-02340eb42995 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.935144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e6e28-e818-4581-b1fc-02340eb42995 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.935441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e6e28-e818-4581-b1fc-02340eb42995 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.937177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 302/594] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:07.943004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91b2c-0e5d-4b7f-a3d8-9c5a5624dff7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.943343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91b2c-0e5d-4b7f-a3d8-9c5a5624dff7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.943586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91b2c-0e5d-4b7f-a3d8-9c5a5624dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.958936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:07.960005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:07.960545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:07.961180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:07.962021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-91b91b2c-0e5d-4b7f-a3d8-9c5a5624dff7 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:07.962140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:07.963256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-91b91b2c-0e5d-4b7f-a3d8-9c5a5624dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:07.964161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:07.975506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 293/595] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:07 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:08.042535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 303/596] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:08.315789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:08.388840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:08.438500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa963b-825d-4a57-bd85-fcaf1d0194eb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:08.487919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:08.488552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:08.488832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:08.489108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:08.504265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:08.509978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:08.510884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:08.512147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 294/597] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:08.517204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b91b2c-0e5d-4b7f-a3d8-9c5a5624dff7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:08.545003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:08.620892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b91b2c-0e5d-4b7f-a3d8-9c5a5624dff7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:08.659490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:08.660105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:08.660384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:08.660653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:08.661311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:08.661608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:08.662975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:08.672614 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 304/598] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:51:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:08.678328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:08.784901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd27b76-3350-4823-a449-d2cb18233d76 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:08.819390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:08.833483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:08.834157 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:08.834437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:08.834707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:08.850066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:08.851593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:08.868593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:08.869554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:08.871072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:08.871072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:08.871072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:08.871072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 295/599] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:08.875363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:08.969220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:08.983488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638145cd-0120-4550-a902-efcb4c17a55c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:09.019526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376be30e-ec5b-433d-a5d2-414bfe606e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:09.020121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376be30e-ec5b-433d-a5d2-414bfe606e3e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.020397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376be30e-ec5b-433d-a5d2-414bfe606e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.020670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376be30e-ec5b-433d-a5d2-414bfe606e3e None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.021300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376be30e-ec5b-433d-a5d2-414bfe606e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:09.021600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376be30e-ec5b-433d-a5d2-414bfe606e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:09.023546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376be30e-ec5b-433d-a5d2-414bfe606e3e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:09.036134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 305/600] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:09.042232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:09.078718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc167c1-fb80-44f5-8707-d00449f8e577 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.079031 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc167c1-fb80-44f5-8707-d00449f8e577 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.079310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc167c1-fb80-44f5-8707-d00449f8e577 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.080709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 306/601] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:09.087404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cece0-235d-4fed-bd19-5370f98ae2aa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.087680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cece0-235d-4fed-bd19-5370f98ae2aa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.087947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cece0-235d-4fed-bd19-5370f98ae2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.113146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-120cece0-235d-4fed-bd19-5370f98ae2aa None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:09.114717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-120cece0-235d-4fed-bd19-5370f98ae2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:09.197018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 307/602] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:09.272438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c804be7-9c7c-474f-a633-a036d44c42d8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:09.329719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:09.330344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.330618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.330886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.344239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:09.345684 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:09.360492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:09.361457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:09.363000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 296/603] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:51:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:09.368294 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cece0-235d-4fed-bd19-5370f98ae2aa None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:09.480459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:09.508245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:09.539182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:09.549357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cece0-235d-4fed-bd19-5370f98ae2aa None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:09.617676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ca3de-aa4e-436f-a51a-cda2ad36e9a1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:09.621029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:09.621667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.621952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.622226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.622949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:09.630904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:09.630904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:09.662305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 308/604] 10.222.0.36 () {56 vars in 1149 bytes} [Mon Oct 17 06:51:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:09.663232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:09.688646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:09.689294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.689545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.689821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.710203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:09.719108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:09.720082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:09.720910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:09.722316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 297/605] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:09.728346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:09.799125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a813-9e4c-430b-9c73-611e0aa01d27 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:09.834754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b241a-06a5-4d19-809b-9d165d22c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:09.836454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b241a-06a5-4d19-809b-9d165d22c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.836950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b241a-06a5-4d19-809b-9d165d22c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.837771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b241a-06a5-4d19-809b-9d165d22c1f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.842324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:09.911870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:09.922066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d808dd-2fc2-4749-a942-c6b07b7bf96f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:09.972434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b241a-06a5-4d19-809b-9d165d22c1f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:09.973886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b241a-06a5-4d19-809b-9d165d22c1f1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:09.996870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:09.997501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:09.997778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:09.998049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:09.998727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:09.999034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:10.000511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:10.026062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 298/606] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:10.031849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.172766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 309/607] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:51:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:51:10.179017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c8de8-bd58-4e4a-9217-78cd5e37893b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.179572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c8de8-bd58-4e4a-9217-78cd5e37893b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.180078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c8de8-bd58-4e4a-9217-78cd5e37893b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.181768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 310/608] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:10.182631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c8de8-bd58-4e4a-9217-78cd5e37893b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.203756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71506d1-164d-4c7d-aa3c-c4e129caf10a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:10.227426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:10.248936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ead7cb-6ab4-4027-a3f0-19ba43422770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:10.250475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead7cb-6ab4-4027-a3f0-19ba43422770 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.250978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead7cb-6ab4-4027-a3f0-19ba43422770 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.251768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead7cb-6ab4-4027-a3f0-19ba43422770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.265245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ead7cb-6ab4-4027-a3f0-19ba43422770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:10.267363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ead7cb-6ab4-4027-a3f0-19ba43422770 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-16ead7cb-6ab4-4027-a3f0-19ba43422770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:10.285653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:10.286733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:10.287689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:10.287689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:10.287689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:10.287689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 299/609] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:10.293242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4591a0-13df-41d0-a5ab-ca8301ba89f7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.293242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4591a0-13df-41d0-a5ab-ca8301ba89f7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.293242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4591a0-13df-41d0-a5ab-ca8301ba89f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.310492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c8de8-bd58-4e4a-9217-78cd5e37893b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:10.311442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eb4591a0-13df-41d0-a5ab-ca8301ba89f7 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:10.313207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eb4591a0-13df-41d0-a5ab-ca8301ba89f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:10.317046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:10.351529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e68090-8512-482a-819e-81521a288301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:10.352605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e68090-8512-482a-819e-81521a288301 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.353076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e68090-8512-482a-819e-81521a288301 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.353536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e68090-8512-482a-819e-81521a288301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.364517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e68090-8512-482a-819e-81521a288301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:10.366288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e68090-8512-482a-819e-81521a288301 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-22e68090-8512-482a-819e-81521a288301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:10.382191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:10.383193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:10.384165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:10.384165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:10.384165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:10.385939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 311/610] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:10.386911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e68090-8512-482a-819e-81521a288301 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.415088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 300/611] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:10.415088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4591a0-13df-41d0-a5ab-ca8301ba89f7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.549837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4591a0-13df-41d0-a5ab-ca8301ba89f7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:10.552595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:10.575342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e68090-8512-482a-819e-81521a288301 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:10.624130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:10.624741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.625019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.625292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.625932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:10.626232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:10.631461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:10.644675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:10.645281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.645557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.645841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.646585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:10.646882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:10.650256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:10.652024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:10.659848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 301/612] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:10.666806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.684908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 312/613] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:10.691041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.818042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2fff1-f76e-4dd9-990b-4a7a0d2c96d6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:10.824879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c989-f5df-428a-8e9a-200b376ee28c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:10.851846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:10.852467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.852739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.853004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.859574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:10.859801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d0b9f-2ddb-4f01-bd0e-5a9fc39e75e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:10.860415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d0b9f-2ddb-4f01-bd0e-5a9fc39e75e2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.860691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d0b9f-2ddb-4f01-bd0e-5a9fc39e75e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.860958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d0b9f-2ddb-4f01-bd0e-5a9fc39e75e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.861050 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:10.867775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d0b9f-2ddb-4f01-bd0e-5a9fc39e75e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:10.869182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d0b9f-2ddb-4f01-bd0e-5a9fc39e75e2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:10.872542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:10.873412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:10.874291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 302/614] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:10.874788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-6e7d0b9f-2ddb-4f01-bd0e-5a9fc39e75e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:10.881511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:10.883025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 313/615] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2876db2-44ae-4278-8fd8-0b1084fc6494 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.884648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2876db2-44ae-4278-8fd8-0b1084fc6494 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.885813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2876db2-44ae-4278-8fd8-0b1084fc6494 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.887273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 314/616] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:10.887273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a141274-e1da-4241-9c8c-b2d43a7fc457 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:10.887522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a141274-e1da-4241-9c8c-b2d43a7fc457 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:10.887869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a141274-e1da-4241-9c8c-b2d43a7fc457 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:10.888918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 315/617] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:10.889598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a141274-e1da-4241-9c8c-b2d43a7fc457 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:10.900707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:10.938366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:10.979540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476998d-db65-4652-a3d9-0320131fe623 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:10.989342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a141274-e1da-4241-9c8c-b2d43a7fc457 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.012253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.012869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.013143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.013415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] PATH_INFO: `/v3/services/network` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.013980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.015395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.021506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c629422-1e5c-483d-aca9-46b3d87a56ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.022088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c629422-1e5c-483d-aca9-46b3d87a56ec None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.022362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c629422-1e5c-483d-aca9-46b3d87a56ec None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.022624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c629422-1e5c-483d-aca9-46b3d87a56ec None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.023370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c629422-1e5c-483d-aca9-46b3d87a56ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:11.023666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c629422-1e5c-483d-aca9-46b3d87a56ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:11.023887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:11.025087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:51:11.026120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:11.027072 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c629422-1e5c-483d-aca9-46b3d87a56ec None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.029697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 303/618] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:11.030498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.037030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 316/619] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:11.038194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f22d5-5e3b-4170-a478-4f7d637220c0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.038984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f22d5-5e3b-4170-a478-4f7d637220c0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.039343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f22d5-5e3b-4170-a478-4f7d637220c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.057433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-470f22d5-5e3b-4170-a478-4f7d637220c0 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:11.058615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-470f22d5-5e3b-4170-a478-4f7d637220c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:11.118322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.123869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 317/620] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:11.125104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d869a-db7a-495d-816d-4c9285c1bf57 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.125398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d869a-db7a-495d-816d-4c9285c1bf57 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.125680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d869a-db7a-495d-816d-4c9285c1bf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.142514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bb6d869a-db7a-495d-816d-4c9285c1bf57 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:11.145616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb6d869a-db7a-495d-816d-4c9285c1bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:11.145764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082a18-93e0-42de-adce-0b253fd1811b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.170176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.181921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.182901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.183583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.183855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.184569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:11.184867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.186322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.207943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 304/621] 10.222.0.36 () {56 vars in 1135 bytes} [Mon Oct 17 06:51:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:11.208724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.219696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 318/622] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:11.220828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d869a-db7a-495d-816d-4c9285c1bf57 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.335771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d869a-db7a-495d-816d-4c9285c1bf57 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.341154 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79701-58e9-4888-bb20-842c83fec642 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.364070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.374144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.374997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.375707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.376173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.377069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] RBAC: Adding query filter params (role.id=f2f75648d0384a4cac1041775239be27, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=96d71699c4c648529f5c6a9dd1ef6ca3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:11.377557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.379195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.381955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.383112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.383484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.383977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.384780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:11.385156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.392318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.393039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 319/624] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:51:11 2022] GET /identity/v3/role_assignments?user.id=96d71699c4c648529f5c6a9dd1ef6ca3&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=f2f75648d0384a4cac1041775239be27 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:11.394376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.395293 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 305/624] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:11.396736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.450366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.513012 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f704d43-b5d2-46fd-9e74-e60c49a4e3a8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.540891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b396d4f-c668-4f57-8223-66614c9cd429 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.549317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.549691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.549970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.550244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.550872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] RBAC: Adding query filter params (type=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:11.551202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.552646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.563279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 306/625] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:11.563279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.588326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.588939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.589784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.589784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.652774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.665069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.666516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.679965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfe9ba-ac86-480f-938e-fd5f8dac7e36 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.691090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.717817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.718419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.718697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.718972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.719628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.720987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.736922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 307/626] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:11 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:11.737760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.804016 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 320/627] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:51:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:51:11.805340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.886441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50658b2-3780-473a-89c8-bcff7f1f29c8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.927891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.928615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:11.929478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:11.929857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:11.930442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:11.931666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.931843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:11.939709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df85e3c-c310-4436-936d-3c30e2c10000 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:11.941525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:11.942489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:11.944615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 308/628] 10.222.0.36 () {56 vars in 1129 bytes} [Mon Oct 17 06:51:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:11.946876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:11.981472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:11.999128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:11.999938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.000342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.000627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.001321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.003445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.033386 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 321/629] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:12.034235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:12.065398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeca4d-94b9-4509-86de-2debdb1a0710 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.110300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.111003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.111329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.111604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.123809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.125325 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:12.142992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:12.144145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:12.145373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:12.145373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:12.145373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:12.145373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 309/630] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:12.151323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:12.176355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ebe62-4c3f-4484-954d-3ec2af72024d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.259720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.260818 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.261341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.261613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.262258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:12.262560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.263987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.277638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 322/631] 10.222.0.36 () {56 vars in 1124 bytes} [Mon Oct 17 06:51:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:12.284876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:12.290932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:12.311418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:12.326085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b4e72-1da4-44c3-b1db-cf5ced38d433 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.374379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.376077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.376864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.377582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.378421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:12.379247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.381928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.399975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 310/632] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:12.405137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:12.430995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e69b9-98b7-47e3-8a25-5d8ce1cc685b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.477838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.478448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.478726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.478999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.479644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:12.480202 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.482136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.488505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 323/633] 10.222.0.36 () {56 vars in 1124 bytes} [Mon Oct 17 06:51:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:12.494555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:12.533163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67f655-a314-4f9d-9791-55e6f428d2c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.579821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.580810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.581095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.581370 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.582803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:12.591265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.593286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.595132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:12.609989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:12.611079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:12.612077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 311/634] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:12.616992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:12.618639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff90565-6e84-4510-9a86-b5ca10fc7019 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.654565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.655155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.655466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.655736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.656395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.657984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.681255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 324/635] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:51:12.746111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460114b-27e7-4ff2-9a20-80ab8c99a6a0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.794929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dd7bc-44c6-4cc9-9a52-a8920ea7ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.795642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dd7bc-44c6-4cc9-9a52-a8920ea7ca15 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.795925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dd7bc-44c6-4cc9-9a52-a8920ea7ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.796250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dd7bc-44c6-4cc9-9a52-a8920ea7ca15 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.797000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dd7bc-44c6-4cc9-9a52-a8920ea7ca15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:12.797346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dd7bc-44c6-4cc9-9a52-a8920ea7ca15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.798816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dd7bc-44c6-4cc9-9a52-a8920ea7ca15 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.814213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 312/636] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:12.820245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:12.927682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db97f3-ecf5-446c-a9d0-b22e03424eeb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:12.956829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:12.962441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf4790-f58c-41e9-a3fb-03611cd0098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:12.963186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4790-f58c-41e9-a3fb-03611cd0098c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:12.963491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4790-f58c-41e9-a3fb-03611cd0098c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:12.963767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4790-f58c-41e9-a3fb-03611cd0098c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:12.964477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4790-f58c-41e9-a3fb-03611cd0098c None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=5f4baa1481254765b8af7a253806f28f) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:12.964847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4790-f58c-41e9-a3fb-03611cd0098c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:12.966402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4790-f58c-41e9-a3fb-03611cd0098c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:12.972663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 325/637] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:51:12 2022] GET /identity/v3/role_assignments?user.id=5f4baa1481254765b8af7a253806f28f&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:13.703987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251f034-8c44-4ad9-9c85-ccf7a0d2b7f7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:13.703987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251f034-8c44-4ad9-9c85-ccf7a0d2b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:13.703987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251f034-8c44-4ad9-9c85-ccf7a0d2b7f7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:13.703987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 313/638] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:13.713743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763ded1-fe3d-4a12-94a7-b60cefcabebc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:13.713743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763ded1-fe3d-4a12-94a7-b60cefcabebc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:13.713743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763ded1-fe3d-4a12-94a7-b60cefcabebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:13.730616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8763ded1-fe3d-4a12-94a7-b60cefcabebc None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:13.732167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8763ded1-fe3d-4a12-94a7-b60cefcabebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:13.804745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 326/639] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:13 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:14.070198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251f034-8c44-4ad9-9c85-ccf7a0d2b7f7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.106797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93549-8098-45f6-8a31-19ab3a834eb2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.107081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93549-8098-45f6-8a31-19ab3a834eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.107381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93549-8098-45f6-8a31-19ab3a834eb2 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.108980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 327/640] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:14.113969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a3406-2353-440c-88f2-970f713ae67d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.114252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a3406-2353-440c-88f2-970f713ae67d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.114632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a3406-2353-440c-88f2-970f713ae67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.133196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:14.140678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d76a3406-2353-440c-88f2-970f713ae67d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:14.142621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d76a3406-2353-440c-88f2-970f713ae67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:14.189068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251f034-8c44-4ad9-9c85-ccf7a0d2b7f7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:14.225375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:14.226498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.227016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.227528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.231732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 328/641] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:14 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:14.242708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:14.247763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:14.248679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:14.249511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:14.252090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 314/642] 10.222.0.36 () {56 vars in 1120 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:14.258075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a3406-2353-440c-88f2-970f713ae67d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.314512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:14.363999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a3406-2353-440c-88f2-970f713ae67d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:14.399466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:14.400069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.400345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.400615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.401263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:14.401562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:14.402911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:14.411241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 329/643] 10.222.0.36 () {56 vars in 1115 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:14.417185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.497407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.521145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04351aa6-10aa-4f39-83e7-52483a3d9b67 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:14.543420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:14.570993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:14.571727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.572261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.572542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.588117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:14.589593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:14.611655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:14.612706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:14.613588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:14.613588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:14.613588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:14.615344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 315/644] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:14.618458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.628617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bc564-e2ee-4738-86c4-e610b4e23790 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:14.650764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:14.671656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:14.672309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.672584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.672849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.673414 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:14.675400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:14.685834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:14.687095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:14.688056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 330/645] 10.222.0.36 () {56 vars in 1132 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:14.696162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.744948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89e9ba-c794-47a4-9bc7-3d2ac2269e95 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:14.785828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:14.786448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.786725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.786990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.787646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:14.788000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:14.789407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:14.806894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 316/646] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:14.808092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.827608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942d4b-b60b-4d4d-a1fc-a34fdc1687ad None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:14.841075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:14.871234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:14.872093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.872539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.872968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.874053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:14.874519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:14.876938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:14.888942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 331/647] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:14.890207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:14.923381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275842cb-c0fc-434c-b988-85e74fcd7480 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:14.949883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:14.959786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f2784-3d6b-4bd9-bf93-ca5ef79868ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:14.961145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f2784-3d6b-4bd9-bf93-ca5ef79868ee None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.961495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f2784-3d6b-4bd9-bf93-ca5ef79868ee None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.961858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f2784-3d6b-4bd9-bf93-ca5ef79868ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.970372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f2784-3d6b-4bd9-bf93-ca5ef79868ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:14.972146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f2784-3d6b-4bd9-bf93-ca5ef79868ee None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-277f2784-3d6b-4bd9-bf93-ca5ef79868ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:14.984513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:14.985574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:14.986536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 317/648] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:14.993031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370522c-6355-447a-8477-2a70aac2a232 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:14.993031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370522c-6355-447a-8477-2a70aac2a232 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:14.993031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370522c-6355-447a-8477-2a70aac2a232 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:14.993031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 318/649] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:14.993031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370522c-6355-447a-8477-2a70aac2a232 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.017646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c54730-918b-4614-bb76-fc0ecf5b4de8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.057362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb02bd-14f7-4912-8d4a-e3f535a0fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.057986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb02bd-14f7-4912-8d4a-e3f535a0fa0f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.058336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb02bd-14f7-4912-8d4a-e3f535a0fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.058521 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb02bd-14f7-4912-8d4a-e3f535a0fa0f None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.059315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb02bd-14f7-4912-8d4a-e3f535a0fa0f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:15.059495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb02bd-14f7-4912-8d4a-e3f535a0fa0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.060911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb02bd-14f7-4912-8d4a-e3f535a0fa0f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.066726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 332/650] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:15.068014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2282-3d48-4771-9f9c-2f95830c28c2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.068296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2282-3d48-4771-9f9c-2f95830c28c2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.068565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2282-3d48-4771-9f9c-2f95830c28c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.087841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4f9b2282-3d48-4771-9f9c-2f95830c28c2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:15.089015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4f9b2282-3d48-4771-9f9c-2f95830c28c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:15.094581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:15.111019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370522c-6355-447a-8477-2a70aac2a232 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.148659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296e645-a9ec-491c-88b8-12927f5a130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.149227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296e645-a9ec-491c-88b8-12927f5a130d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.149500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296e645-a9ec-491c-88b8-12927f5a130d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.149763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296e645-a9ec-491c-88b8-12927f5a130d None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.150523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296e645-a9ec-491c-88b8-12927f5a130d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:15.150827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296e645-a9ec-491c-88b8-12927f5a130d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.152308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296e645-a9ec-491c-88b8-12927f5a130d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.167706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 333/651] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:15.168535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b2282-3d48-4771-9f9c-2f95830c28c2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.170659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 319/652] 10.222.0.36 () {56 vars in 1149 bytes} [Mon Oct 17 06:51:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:15.171892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7774d3-c280-4e10-bd18-2b3e75c81b0e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.172171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7774d3-c280-4e10-bd18-2b3e75c81b0e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.172434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7774d3-c280-4e10-bd18-2b3e75c81b0e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.173770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 320/653] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:15.176020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7774d3-c280-4e10-bd18-2b3e75c81b0e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.219528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:15.281392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b2282-3d48-4771-9f9c-2f95830c28c2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.284011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7774d3-c280-4e10-bd18-2b3e75c81b0e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.319025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.320091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665048e-f5bf-4de1-92e7-5e13a540540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.320212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.320445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.320692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665048e-f5bf-4de1-92e7-5e13a540540a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.320798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.320968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665048e-f5bf-4de1-92e7-5e13a540540a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.321232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665048e-f5bf-4de1-92e7-5e13a540540a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.321909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665048e-f5bf-4de1-92e7-5e13a540540a None admin] RBAC: Adding query filter params (interface=public, service_id=11c2f1cc7c43455e93d0f46eefb3d8e7, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:15.322211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665048e-f5bf-4de1-92e7-5e13a540540a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.323703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665048e-f5bf-4de1-92e7-5e13a540540a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.335432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 334/654] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/endpoints?service_id=11c2f1cc7c43455e93d0f46eefb3d8e7&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:15.335604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.335699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b3781-11fe-4ee1-8bb4-1c54043426e9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.335699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b3781-11fe-4ee1-8bb4-1c54043426e9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.335864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.335948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b3781-11fe-4ee1-8bb4-1c54043426e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.338902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:15.351877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-531b3781-11fe-4ee1-8bb4-1c54043426e9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:15.352732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:15.353639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:15.354540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:15.354540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:15.354540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:15.354709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-531b3781-11fe-4ee1-8bb4-1c54043426e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:15.354789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 321/655] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:15.355364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.431437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 335/656] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:15.431437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b3781-11fe-4ee1-8bb4-1c54043426e9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.457810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5ed92-0f78-4c4e-9724-3eedf6c9c993 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.463936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:15.496192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.496897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.497173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.497468 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.498145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.499543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.515252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 322/657] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/services => generated 1029 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:51:15.515252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.547310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b3781-11fe-4ee1-8bb4-1c54043426e9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.594759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:15.595662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.596456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.596794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.597161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.598111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:15.598481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.600920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.619467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 336/658] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:15.625208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.654165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92997-a4de-44f2-9211-5a9ce919d576 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.702022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.702619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.702900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.703170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.703775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.705229 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:15.714580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:15.715631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:15.716474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 323/659] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:15.717263 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.737035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:15.744966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ab977-49bd-4716-a553-5b5b07cd8656 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.783326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.783953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.784225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.784493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.792426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.793861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:15.806628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:15.807767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 337/660] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:15.813684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.822349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4aa10-315d-4f82-903e-7cfecc7a1255 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.846650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:15.859712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.860342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.860620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.860893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.861465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.862832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:15.871560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:15.872593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:15.873423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 324/661] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:15.874227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.925433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b821d6-eb12-4d53-94ca-8b0f43800d73 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.962248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:15.962782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:15.963062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:15.963366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:15.964019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:15.964396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:15.965889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:15.971358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 338/662] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:15.972139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:15.978954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32686931-0219-460c-87bb-3777daaeab97 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:15.993509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:16.015846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:16.016445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:16.016717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:16.016982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:16.017690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:16.017950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:16.019401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:16.033381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 325/663] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:16.034187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:16.085927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:16.091688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e23239-7bb2-4614-8a84-0d12e3cbb166 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:16.127758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:16.128357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:16.128636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:16.128903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:16.129517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:16.129815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:16.131174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:16.136606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 339/664] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:16.137346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:16.146350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ba2ae-94e9-45af-aa53-2d7560231ae3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:16.181496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:16.182095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:16.182368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:16.182633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:16.183270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:16.183572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:16.184950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:16.189812 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 326/665] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:16.190570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:16.254711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:16.278383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b655d2-d2d6-459e-bc16-5e883963ed58 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:16.317653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8015a-96e9-4134-9163-91ac235add7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:16.318527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8015a-96e9-4134-9163-91ac235add7b None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:16.318902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8015a-96e9-4134-9163-91ac235add7b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:16.319390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8015a-96e9-4134-9163-91ac235add7b None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/96d71699c4c648529f5c6a9dd1ef6ca3/roles/c6bee23227284e0e907145bc459b5db7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:16.322050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8015a-96e9-4134-9163-91ac235add7b None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=96d71699c4c648529f5c6a9dd1ef6ca3, role_id=c6bee23227284e0e907145bc459b5db7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:16.323693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8015a-96e9-4134-9163-91ac235add7b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:16.336217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f730d-eab0-4bba-a653-b01bb57fe87b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:16.343583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 340/666] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:51:16 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/96d71699c4c648529f5c6a9dd1ef6ca3/roles/c6bee23227284e0e907145bc459b5db7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:51:16.357616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:16.386427 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d54197-79ba-4a88-916c-09d3aa7d2cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:16.387282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d54197-79ba-4a88-916c-09d3aa7d2cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:16.387561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d54197-79ba-4a88-916c-09d3aa7d2cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:16.387827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d54197-79ba-4a88-916c-09d3aa7d2cc6 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:16.388467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d54197-79ba-4a88-916c-09d3aa7d2cc6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:16.388767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d54197-79ba-4a88-916c-09d3aa7d2cc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:16.390162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d54197-79ba-4a88-916c-09d3aa7d2cc6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:16.395791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 327/667] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:17.685471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31aac7-caac-454f-9f8c-ace7d9a80993 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:17.686006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31aac7-caac-454f-9f8c-ace7d9a80993 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:17.686006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31aac7-caac-454f-9f8c-ace7d9a80993 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:17.687857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 341/668] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:17.693070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1799-af68-4a92-ab64-8914d6bc6667 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:17.693447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1799-af68-4a92-ab64-8914d6bc6667 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:17.693664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1799-af68-4a92-ab64-8914d6bc6667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:17.712268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bf0f1799-af68-4a92-ab64-8914d6bc6667 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:17.713422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bf0f1799-af68-4a92-ab64-8914d6bc6667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:17.780862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 328/669] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:17 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:18.044619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31aac7-caac-454f-9f8c-ace7d9a80993 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:18.153751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31aac7-caac-454f-9f8c-ace7d9a80993 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:18.189583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:18.190210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.190484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.190751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.191344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:18.192792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:18.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:18.203099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:18.204017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 342/670] 10.222.0.36 () {56 vars in 1132 bytes} [Mon Oct 17 06:51:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:18.209134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f1799-af68-4a92-ab64-8914d6bc6667 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:18.310646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f1799-af68-4a92-ab64-8914d6bc6667 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:18.319629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:18.348987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129eacde-a3ec-420a-8a4b-abdad06cea67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:18.349596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129eacde-a3ec-420a-8a4b-abdad06cea67 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.349868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129eacde-a3ec-420a-8a4b-abdad06cea67 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.350134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129eacde-a3ec-420a-8a4b-abdad06cea67 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.351332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129eacde-a3ec-420a-8a4b-abdad06cea67 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:18.351632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129eacde-a3ec-420a-8a4b-abdad06cea67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:18.353062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129eacde-a3ec-420a-8a4b-abdad06cea67 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:18.358029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 329/671] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:18.364364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:18.374952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:18.438444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b1fd-516d-4d93-9785-67d0beed5c49 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.438734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b1fd-516d-4d93-9785-67d0beed5c49 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.439006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b1fd-516d-4d93-9785-67d0beed5c49 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.440719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 330/672] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:18.447380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87135c54-30ab-4414-aa48-48826492b945 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.447999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87135c54-30ab-4414-aa48-48826492b945 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.448494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87135c54-30ab-4414-aa48-48826492b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.473489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-87135c54-30ab-4414-aa48-48826492b945 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:18.476454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87135c54-30ab-4414-aa48-48826492b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:18.483136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4411fca-cd2d-4045-88c8-61c6c9fdc411 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:18.526608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:18.527238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.527511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.527777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.528393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:18.528685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:18.530042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:18.541692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 343/673] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:18.542942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:18.566285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 331/674] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:18 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:18.567955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d4675-4632-4818-b639-e32431ba2294 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.568455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d4675-4632-4818-b639-e32431ba2294 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.568928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d4675-4632-4818-b639-e32431ba2294 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.570766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 332/675] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:18.577446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62479e-97d9-4812-8911-c6b6b951d5e1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.578044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62479e-97d9-4812-8911-c6b6b951d5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.578526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62479e-97d9-4812-8911-c6b6b951d5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.597854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ff62479e-97d9-4812-8911-c6b6b951d5e1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:18.599353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ff62479e-97d9-4812-8911-c6b6b951d5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:18.618708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:18.650462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e73670-d2df-4021-8654-73c90cd178a9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:18.671883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 333/676] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:18 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:18.673156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297603c3-c213-43de-8df6-4db781bdd250 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.673433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297603c3-c213-43de-8df6-4db781bdd250 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.673700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297603c3-c213-43de-8df6-4db781bdd250 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.674905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 334/677] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:18.680294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6850e28-323e-4e1f-9210-5a55622448ca None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.680788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6850e28-323e-4e1f-9210-5a55622448ca None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.681063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6850e28-323e-4e1f-9210-5a55622448ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.693780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:18.695807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:18.696287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:18.696796 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:18.697655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:18.699804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:18.704209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a6850e28-323e-4e1f-9210-5a55622448ca None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:18.705555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a6850e28-323e-4e1f-9210-5a55622448ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:18.731233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 344/678] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:51:18.744418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:18.802435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 335/679] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:18 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:18.863979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:18.938119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6850e28-323e-4e1f-9210-5a55622448ca None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:18.982817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7ef02-c1ea-46d0-8aba-208aef8d5f57 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:19.014735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:19.032207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:19.032813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:19.033091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:19.033364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:19.034013 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:19.035411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:19.053920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 345/680] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:18 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:19.073735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6850e28-323e-4e1f-9210-5a55622448ca None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:19.074570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:19.122548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:19.123523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:19.123946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:19.124495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:19.125292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:19.126863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:19.155607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 336/681] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:18 2022] POST /identity/v3/services => generated 241 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:19.216972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa1f6e-424d-4d10-84ef-988b1124b831 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:19.264388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:19.265099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:19.265381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:19.265651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:19.280205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:19.285139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:19.286142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:19.287001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 346/682] 10.222.0.36 () {56 vars in 1120 bytes} [Mon Oct 17 06:51:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:19.293099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:19.318827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:19.402944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d78dc-fc5b-41bb-a946-8a40cc701567 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:19.437840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:19.438811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:19.439309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:19.439793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:19.440646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:19.441138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:19.442700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:19.451698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 337/683] 10.222.0.36 () {56 vars in 1115 bytes} [Mon Oct 17 06:51:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:19.460879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:19.508205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:19.566586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2034d7-ff7a-4266-b2b5-e98d83536f86 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:19.601306 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:19.601979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:19.602258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:19.602526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:19.613399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:19.614862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:19.629511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:19.630462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:19.631371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:19.631371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:19.631371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:19.631371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 347/684] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:19.636973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:19.718337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:19.746420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f960935-d0ac-4b2a-8138-5f528416d801 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:19.781022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:19.781620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:19.781899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:19.782169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:19.782802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:19.783101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:19.784490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:19.797577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 338/685] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:19.803097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:19.900394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:19.907468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f1c64-db23-478c-a822-2f71a9f8ca11 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:19.942990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:19.944019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:19.945027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:19.945505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:19.953068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:19.954901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:19.966784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:19.967767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:51:19.968788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:19.975910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 348/686] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:51:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:19.980292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:20.090080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befe272-7b1b-47aa-a75d-da3bd55cd487 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:20.124985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:20.125591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:20.125864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:20.126131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:20.126841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:20.127138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:20.128584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:20.130305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:20.146958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 339/687] 10.222.0.36 () {56 vars in 1149 bytes} [Mon Oct 17 06:51:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:20.153313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:20.255717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eab6f-e149-4fe0-b002-1aa66944f017 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:20.288631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:20.289229 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:20.289503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:20.289767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:20.315513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:20.357751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:20.359187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:20.457522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 349/688] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:51:20 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:51:20.463197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:20.566078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d15881-6f9c-4829-829e-f5efe79f96d0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:20.601536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:20.602156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:20.602429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:20.602711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:20.612835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:20.614268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:20.629133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:20.630113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:20.631001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:20.631001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:20.631001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:20.631001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 340/689] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:20.636690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:20.691272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:20.735246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600375f-e5f6-406d-9b2d-f815f7732fbf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:20.769382 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:20.769982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:20.770255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:20.770568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:20.771195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:20.771520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:20.772940 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:20.784276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 350/690] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:20.790621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:20.840854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:20.902723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cfa57-64ec-45fd-8ad7-b88cdd20230b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:20.939939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d9ead-d3fc-4967-9bd8-405ba61b33fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:20.940565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d9ead-d3fc-4967-9bd8-405ba61b33fb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:20.940935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d9ead-d3fc-4967-9bd8-405ba61b33fb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:20.941210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d9ead-d3fc-4967-9bd8-405ba61b33fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:20.948076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d9ead-d3fc-4967-9bd8-405ba61b33fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:20.949512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d9ead-d3fc-4967-9bd8-405ba61b33fb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-314d9ead-d3fc-4967-9bd8-405ba61b33fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:20.961394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:20.963420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:20.965510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 341/691] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:20.968219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:21.005945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b548-29df-4fa0-bf4d-ed81049265b0 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.006293 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b548-29df-4fa0-bf4d-ed81049265b0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.006485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b548-29df-4fa0-bf4d-ed81049265b0 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.009219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 342/692] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:21.013216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233a30b-b849-4c09-8490-f2d4290dc53e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.013500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233a30b-b849-4c09-8490-f2d4290dc53e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.013773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233a30b-b849-4c09-8490-f2d4290dc53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.035983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1233a30b-b849-4c09-8490-f2d4290dc53e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:21.037419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1233a30b-b849-4c09-8490-f2d4290dc53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:21.077748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b0f3f-bc91-49d7-a60b-6db62b4f5ee0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:21.095103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:21.122795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ad55-ef5d-4c1c-8ff9-143d44938c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:21.123424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ad55-ef5d-4c1c-8ff9-143d44938c1f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.123700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ad55-ef5d-4c1c-8ff9-143d44938c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.123970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ad55-ef5d-4c1c-8ff9-143d44938c1f None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.124640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44ad55-ef5d-4c1c-8ff9-143d44938c1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:21.124941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44ad55-ef5d-4c1c-8ff9-143d44938c1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:21.126356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44ad55-ef5d-4c1c-8ff9-143d44938c1f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:21.137458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 343/693] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:21.144992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 351/694] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:21.156515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233a30b-b849-4c09-8490-f2d4290dc53e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:21.191360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:21.266574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233a30b-b849-4c09-8490-f2d4290dc53e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:21.304313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284b3a8-85bc-4cd0-8e57-0d029782fda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:21.304915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284b3a8-85bc-4cd0-8e57-0d029782fda5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.305194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284b3a8-85bc-4cd0-8e57-0d029782fda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.305461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284b3a8-85bc-4cd0-8e57-0d029782fda5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.306168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284b3a8-85bc-4cd0-8e57-0d029782fda5 None admin] RBAC: Adding query filter params (role.id=c6bee23227284e0e907145bc459b5db7, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=96d71699c4c648529f5c6a9dd1ef6ca3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:21.306470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284b3a8-85bc-4cd0-8e57-0d029782fda5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:21.307914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284b3a8-85bc-4cd0-8e57-0d029782fda5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:21.314276 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 344/695] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:51:21 2022] GET /identity/v3/role_assignments?user.id=96d71699c4c648529f5c6a9dd1ef6ca3&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=c6bee23227284e0e907145bc459b5db7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:21.377435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dccc-e5d8-410e-85b1-8381927f9948 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.377692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dccc-e5d8-410e-85b1-8381927f9948 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.377885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dccc-e5d8-410e-85b1-8381927f9948 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.379466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 352/696] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:21.385785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294490e-24b1-4def-a681-6ec5929fedf4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.386342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294490e-24b1-4def-a681-6ec5929fedf4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.386801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294490e-24b1-4def-a681-6ec5929fedf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.406483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f294490e-24b1-4def-a681-6ec5929fedf4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:21.408073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f294490e-24b1-4def-a681-6ec5929fedf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:21.412558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3dccc-e5d8-410e-85b1-8381927f9948 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:21.508711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 345/697] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:21.539888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3dccc-e5d8-410e-85b1-8381927f9948 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:21.570174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13337979-1ed8-4d16-975e-38cfe7eb24fc None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.570494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13337979-1ed8-4d16-975e-38cfe7eb24fc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.570762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13337979-1ed8-4d16-975e-38cfe7eb24fc None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.572341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 346/698] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:21.576964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06d8f9-97e6-4c5a-866b-586799db600e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.577240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06d8f9-97e6-4c5a-866b-586799db600e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.577503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06d8f9-97e6-4c5a-866b-586799db600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.581243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:21.581962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.582234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.582501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.583071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:21.584541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:21.594333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:21.595374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:21.596199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 353/699] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:21.601406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:21.602731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fb06d8f9-97e6-4c5a-866b-586799db600e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:21.604429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fb06d8f9-97e6-4c5a-866b-586799db600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:21.635884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:21.646728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:21.692418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 347/700] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:21.715983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379ec14-f4bd-48bf-9b9c-ab95e78c58cf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:21.751346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:21.752172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.752624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.753073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.753820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:21.754258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:21.755775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:21.761107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 354/701] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:21.771250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06d8f9-97e6-4c5a-866b-586799db600e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:21.806847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:21.890366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06d8f9-97e6-4c5a-866b-586799db600e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:21.921449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e2374-de9e-4035-9f61-c535379a7b67 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:21.923465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:21.941031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:21.942593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.943092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.943558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.944866 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:21.945370 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:21.947580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:21.959244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 348/702] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:21.969170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:21.969853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:21.970082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:21.970349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:21.970977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:21.972394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:21.982715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:21.983846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:21.985424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 355/703] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:21.986213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:21.990933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.012731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:22.109438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89277791-b93a-4c67-85fd-877b18687f26 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.123959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb95cf0-47f7-468c-906a-a8137bfe53d4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.154697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.155463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.155740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.156008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.156829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.158163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:22.169898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:22.170951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:22.171855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:22.174194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 349/704] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:22.180049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.184687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.185300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.185573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.185839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.186713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:22.187008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.189127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.194918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 356/705] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:22.200258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.233011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:22.299630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:22.313140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bde78-beef-4f4a-8f70-eb90087e17ab None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.332417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9a810-bfc7-4fc8-8bde-b2ee56099903 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.351993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.352370 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.352646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.352910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.353520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:22.353816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.355171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.363534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 350/706] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:22.366495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.378585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.379999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.380751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.381219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.382363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:22.383389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.385525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.397958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 357/707] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:22.398612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.476062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1c7b-2cf4-48aa-8b05-ebe4a87aacd7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.510136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.510730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.511000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.511300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.511918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:22.512214 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.513636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.523482 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 351/708] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:22.525110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.526675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:22.535605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee9ffe-e538-461a-8551-b99075729f73 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.543518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:22.576980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.577612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.577943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.578208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.578898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] RBAC: Adding query filter params (interface=public, service_id=405816fea8b1427787c1d98eee3256ab, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:22.579275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.580806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.590994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 358/709] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:51:22 2022] GET /identity/v3/endpoints?service_id=405816fea8b1427787c1d98eee3256ab&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:22.592004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.650930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272de8b-d5b8-415c-a732-8cb91a2d82da None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.685501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.686134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.686662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.686929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.687656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] RBAC: Adding query filter params (interface=public, service_id=f2828796c3c8462f88bfd67e75d451a7, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:22.687959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.689323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.695048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 352/710] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:51:22 2022] GET /identity/v3/endpoints?service_id=f2828796c3c8462f88bfd67e75d451a7&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:22.700755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:22.723494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6c47-bfa8-49c7-82ba-5b7c6c246fad None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.770336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135206d-67f0-49c2-abcb-d7cc2c21b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.771557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135206d-67f0-49c2-abcb-d7cc2c21b7ec None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.772035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135206d-67f0-49c2-abcb-d7cc2c21b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.772771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135206d-67f0-49c2-abcb-d7cc2c21b7ec None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.774469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135206d-67f0-49c2-abcb-d7cc2c21b7ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.776067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135206d-67f0-49c2-abcb-d7cc2c21b7ec None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.782340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:22.791918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 359/711] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:22 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 06:51:22.812794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811ae2-1272-4984-a890-77a8b08b279e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:22.851596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e609965-16c0-4a29-992e-b8100dd13aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:22.852210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609965-16c0-4a29-992e-b8100dd13aac None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:22.852481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609965-16c0-4a29-992e-b8100dd13aac None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:22.852745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609965-16c0-4a29-992e-b8100dd13aac None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:22.853346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609965-16c0-4a29-992e-b8100dd13aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:22.854742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609965-16c0-4a29-992e-b8100dd13aac None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:22.862054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 353/712] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:22 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:51:23.573299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52c7c-e0be-4acd-9075-f0fca7adaa72 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:23.573793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52c7c-e0be-4acd-9075-f0fca7adaa72 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:23.573793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52c7c-e0be-4acd-9075-f0fca7adaa72 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:23.576486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 360/713] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:23.580689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910373d4-d449-43d7-8b5c-d262d35b809f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:23.581100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910373d4-d449-43d7-8b5c-d262d35b809f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:23.581272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910373d4-d449-43d7-8b5c-d262d35b809f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:23.599625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-910373d4-d449-43d7-8b5c-d262d35b809f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:23.601433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-910373d4-d449-43d7-8b5c-d262d35b809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:23.670897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 354/714] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:23.938726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a52c7c-e0be-4acd-9075-f0fca7adaa72 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:23.963509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:24.044889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a52c7c-e0be-4acd-9075-f0fca7adaa72 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:24.084266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27543260-b30e-4e35-9da8-fe4cf6ce0aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:24.085744 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27543260-b30e-4e35-9da8-fe4cf6ce0aac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:24.086175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27543260-b30e-4e35-9da8-fe4cf6ce0aac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:24.086592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27543260-b30e-4e35-9da8-fe4cf6ce0aac None admin] PATH_INFO: `/v3/services/image` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:24.087366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27543260-b30e-4e35-9da8-fe4cf6ce0aac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:24.088953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27543260-b30e-4e35-9da8-fe4cf6ce0aac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-27543260-b30e-4e35-9da8-fe4cf6ce0aac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:24.098860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:24.100299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:24.101350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:24.103782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 361/715] 10.222.0.36 () {56 vars in 1105 bytes} [Mon Oct 17 06:51:23 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:24.115261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910373d4-d449-43d7-8b5c-d262d35b809f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:24.185839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88f2b3-bfd0-4c12-9698-0ae61d568fe4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:24.186612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88f2b3-bfd0-4c12-9698-0ae61d568fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:24.187026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88f2b3-bfd0-4c12-9698-0ae61d568fe4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:24.188865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 362/716] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:24.189070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:24.195262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6c372-1bc3-48e2-9b46-e3dcfa4f651b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:24.195408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6c372-1bc3-48e2-9b46-e3dcfa4f651b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:24.195711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6c372-1bc3-48e2-9b46-e3dcfa4f651b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:24.217214 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-75d6c372-1bc3-48e2-9b46-e3dcfa4f651b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:24.218398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-75d6c372-1bc3-48e2-9b46-e3dcfa4f651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:24.220727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910373d4-d449-43d7-8b5c-d262d35b809f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:24.269619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:24.270625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:24.270902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:24.271557 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:24.272238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] RBAC: Adding query filter params (name=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:24.272564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:24.274809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:24.281513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 355/717] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:24.286523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:24.298776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 363/718] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:24.389922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365ebd7-4015-43bf-9a80-4826e4122325 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:24.423808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc8050-fcc6-468c-8682-0853857a20a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:24.424693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc8050-fcc6-468c-8682-0853857a20a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:24.425174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc8050-fcc6-468c-8682-0853857a20a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:24.425636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc8050-fcc6-468c-8682-0853857a20a9 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:24.426507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc8050-fcc6-468c-8682-0853857a20a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:24.427006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc8050-fcc6-468c-8682-0853857a20a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:24.428638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc8050-fcc6-468c-8682-0853857a20a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:24.430784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:24.439817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 356/719] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:24.583443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6c372-1bc3-48e2-9b46-e3dcfa4f651b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:24.601981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:24.689060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6c372-1bc3-48e2-9b46-e3dcfa4f651b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:24.728206 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d1dcd-9760-4247-93c0-9d6f78e57af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:24.729159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1dcd-9760-4247-93c0-9d6f78e57af9 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:24.729631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1dcd-9760-4247-93c0-9d6f78e57af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:24.730094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1dcd-9760-4247-93c0-9d6f78e57af9 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:24.730950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d1dcd-9760-4247-93c0-9d6f78e57af9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:24.732540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d1dcd-9760-4247-93c0-9d6f78e57af9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:24.750362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 364/720] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:24 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:25.066357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e144e0-555c-452a-a33c-22bcfe63f01c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.067398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e144e0-555c-452a-a33c-22bcfe63f01c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.067879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e144e0-555c-452a-a33c-22bcfe63f01c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.070223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 357/721] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:25.075123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c0dce-3499-400e-8678-e4210b64687a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.075443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c0dce-3499-400e-8678-e4210b64687a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.075717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c0dce-3499-400e-8678-e4210b64687a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.097575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-908c0dce-3499-400e-8678-e4210b64687a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:25.099064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-908c0dce-3499-400e-8678-e4210b64687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:25.105472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05617662-99f0-4e2f-884b-7538ef81c205 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.105754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05617662-99f0-4e2f-884b-7538ef81c205 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.106032 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05617662-99f0-4e2f-884b-7538ef81c205 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.107637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 358/722] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:25.112787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6e054-ec84-47e0-8c26-b760eb093251 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.112955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6e054-ec84-47e0-8c26-b760eb093251 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.113151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6e054-ec84-47e0-8c26-b760eb093251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.130966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-55f6e054-ec84-47e0-8c26-b760eb093251 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:25.132837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-55f6e054-ec84-47e0-8c26-b760eb093251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:25.184778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 365/723] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:25.211540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 359/724] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:25.450293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c0dce-3499-400e-8678-e4210b64687a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:25.452738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:25.482021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6e054-ec84-47e0-8c26-b760eb093251 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:25.563410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c0dce-3499-400e-8678-e4210b64687a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:25.600976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6e054-ec84-47e0-8c26-b760eb093251 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:25.603730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:25.604888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.605311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.605740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.606460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:25.607986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:25.619250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:25.620455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:51:25.621415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:25.623736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 366/725] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:25.629059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:25.635834 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:25.643098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:25.644045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.644518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.644982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.645746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:25.647871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:25.656863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:25.657869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:25.658774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:25.661700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 360/726] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:25.668784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:25.711071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:25.764503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0af660-ba6d-466c-a1da-da4f9c053bc2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:25.782105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9978e-9e01-47bc-92ce-d765b5f9a8ce None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:25.809797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:25.810972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.812197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.812679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.813861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:25.814433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:25.816077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:25.817988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:25.818753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.819140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.819526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.820236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:25.820646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:25.822098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:25.823027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 367/727] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:25.828563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:25.837876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 361/728] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:25.839040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:25.920865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:25.934662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd83b3-c190-4089-a021-c8e66939a412 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:25.967353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:25.973774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73767865-506f-402e-9de1-21b44090d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:25.974674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73767865-506f-402e-9de1-21b44090d730 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:25.976005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73767865-506f-402e-9de1-21b44090d730 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:25.977803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08343fcc-adc8-4be3-8ac2-62cbf6dc9cfc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:25.979567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73767865-506f-402e-9de1-21b44090d730 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:25.980197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73767865-506f-402e-9de1-21b44090d730 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:25.980496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73767865-506f-402e-9de1-21b44090d730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:25.981863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73767865-506f-402e-9de1-21b44090d730 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:25.990960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 368/729] 10.222.0.36 () {56 vars in 1114 bytes} [Mon Oct 17 06:51:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:25.996833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73767865-506f-402e-9de1-21b44090d730 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:26.032209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:26.032812 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:26.033084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:26.033346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:26.033957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:26.034254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:26.035645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:26.045984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 362/730] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:26.048035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:26.121190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73767865-506f-402e-9de1-21b44090d730 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:26.168799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb12a-3384-4075-a067-23ae0092cfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:26.170249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb12a-3384-4075-a067-23ae0092cfe4 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:26.170783 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb12a-3384-4075-a067-23ae0092cfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:26.171223 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb12a-3384-4075-a067-23ae0092cfe4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:26.172282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb12a-3384-4075-a067-23ae0092cfe4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:26.174431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb12a-3384-4075-a067-23ae0092cfe4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:26.175621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a734-089d-4907-ab74-9c2fbe4624aa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:26.203293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 369/731] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:51:26.211939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:26.231964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37386ff3-2a1d-420e-9235-f91a323585e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:26.232488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386ff3-2a1d-420e-9235-f91a323585e4 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:26.232813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386ff3-2a1d-420e-9235-f91a323585e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:26.233087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386ff3-2a1d-420e-9235-f91a323585e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:26.233750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37386ff3-2a1d-420e-9235-f91a323585e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:26.235147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37386ff3-2a1d-420e-9235-f91a323585e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:26.261246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 363/732] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:26.680959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439ffd-2d69-44b9-bf37-838f85cec381 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:26.681509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439ffd-2d69-44b9-bf37-838f85cec381 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:26.681509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439ffd-2d69-44b9-bf37-838f85cec381 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:26.682974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 370/733] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:26.688760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ffb7-51bb-42ec-86b3-6307e4683f91 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:26.689273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ffb7-51bb-42ec-86b3-6307e4683f91 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:26.689760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ffb7-51bb-42ec-86b3-6307e4683f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:26.708637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3b74ffb7-51bb-42ec-86b3-6307e4683f91 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:26.710171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3b74ffb7-51bb-42ec-86b3-6307e4683f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:26.791319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 364/734] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:26.958896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff21f-8bf4-4c59-ab49-249df5ca7d91 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:26.959091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff21f-8bf4-4c59-ab49-249df5ca7d91 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:26.959916 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff21f-8bf4-4c59-ab49-249df5ca7d91 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:26.961407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 371/735] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:26.967020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611cd72-92c6-4428-b472-7a34229fdeca None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:26.967584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611cd72-92c6-4428-b472-7a34229fdeca None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:26.968082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611cd72-92c6-4428-b472-7a34229fdeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:26.986524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1611cd72-92c6-4428-b472-7a34229fdeca None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:26.988059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1611cd72-92c6-4428-b472-7a34229fdeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:27.060941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 365/736] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:27.062350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ff21f-8bf4-4c59-ab49-249df5ca7d91 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:27.112690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:27.175520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ff21f-8bf4-4c59-ab49-249df5ca7d91 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:27.209651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:27.210179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:27.210449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:27.210712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:27.211374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:27.212714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:27.231394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 372/737] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:27 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:51:27.333136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611cd72-92c6-4428-b472-7a34229fdeca None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:27.366144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:27.439600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611cd72-92c6-4428-b472-7a34229fdeca None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:27.481632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:27.482233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:27.482513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:27.482785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:27.483436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:27.484932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:27.493942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:27.494931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:27.495798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:27.499314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 366/738] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:27.501471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:27.611589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb6dec-a955-4ea3-8c55-89a11cb751b3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:27.623613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:27.650054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:27.650992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:27.651512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:27.651978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:27.652800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] RBAC: Adding query filter params (name=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:27.653306 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:27.654868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:27.660403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 373/739] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:27.665984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:27.760184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:27.770213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1428629-32fb-4fb4-9008-1860fa47a511 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:27.805332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:27.805990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:27.806275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:27.806546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:27.807230 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:27.807539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:27.808981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:27.814726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 367/740] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:27.820997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:27.920550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca284b4-5bf8-4ee7-9a35-eaa4582902fa None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:27.958902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a428a0-2898-4d57-9e98-68c3e9644e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:27.959955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a428a0-2898-4d57-9e98-68c3e9644e76 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:27.960438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a428a0-2898-4d57-9e98-68c3e9644e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:27.960902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a428a0-2898-4d57-9e98-68c3e9644e76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:27.961790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a428a0-2898-4d57-9e98-68c3e9644e76 None admin] RBAC: Adding query filter params (interface=public, service_id=445affec39894a439a359a8f7d856619, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:27.962391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a428a0-2898-4d57-9e98-68c3e9644e76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:27.963970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a428a0-2898-4d57-9e98-68c3e9644e76 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:27.970003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 374/741] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:51:27 2022] GET /identity/v3/endpoints?service_id=445affec39894a439a359a8f7d856619&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:27.976270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:28.086104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05acb15-a967-458e-806f-8d58d24ca779 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:28.120990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a709ebd-b1d1-4ebb-84bb-3f293317f6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:28.121619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709ebd-b1d1-4ebb-84bb-3f293317f6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:28.121897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709ebd-b1d1-4ebb-84bb-3f293317f6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:28.122249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709ebd-b1d1-4ebb-84bb-3f293317f6b0 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:28.122861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a709ebd-b1d1-4ebb-84bb-3f293317f6b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:28.124322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a709ebd-b1d1-4ebb-84bb-3f293317f6b0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:28.132669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 368/742] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:27 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:51:28.542430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830325-5dca-4223-bace-cd085c511a63 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:28.543131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830325-5dca-4223-bace-cd085c511a63 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:28.543669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830325-5dca-4223-bace-cd085c511a63 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:28.545442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 375/743] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:28.550715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d804f-c375-4dd4-955c-91143689caa7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:28.551565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d804f-c375-4dd4-955c-91143689caa7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:28.552034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d804f-c375-4dd4-955c-91143689caa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:28.571052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bd1d804f-c375-4dd4-955c-91143689caa7 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:28.572553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bd1d804f-c375-4dd4-955c-91143689caa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:28.586807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:28.625623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d5610-c909-4f04-aa7a-83fee94a6df6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:28.625906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d5610-c909-4f04-aa7a-83fee94a6df6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:28.626247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d5610-c909-4f04-aa7a-83fee94a6df6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:28.631228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 376/744] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:28.632884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b744e1a-d1fb-4ffb-b8da-9ae8cce19a9a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:28.633169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b744e1a-d1fb-4ffb-b8da-9ae8cce19a9a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:28.633447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b744e1a-d1fb-4ffb-b8da-9ae8cce19a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:28.645236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 369/745] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:28 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:28.657768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4b744e1a-d1fb-4ffb-b8da-9ae8cce19a9a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:28.659251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4b744e1a-d1fb-4ffb-b8da-9ae8cce19a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:28.690293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:28.728883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 377/746] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:28 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:28.916538 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d804f-c375-4dd4-955c-91143689caa7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.002760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b744e1a-d1fb-4ffb-b8da-9ae8cce19a9a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.027090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d804f-c375-4dd4-955c-91143689caa7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.077276 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.078256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.078988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.079531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.080896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.083046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.087252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING keystone.server.flask.common [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 06:51:29.110521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.sql.core [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 06:51:29.110521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:51:29.110521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [parameters: {'id': 'bf3f7e897cfa4c0c92305e0c54fe20cb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 06:51:29.110521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81488) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:51:29.115006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b744e1a-d1fb-4ffb-b8da-9ae8cce19a9a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf3f7e897cfa4c0c92305e0c54fe20cb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf3f7e897cfa4c0c92305e0c54fe20cb. Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:51:29.121795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 06:51:29.122590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application yield resource Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise self.value Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application session.commit() Oct 17 06:51:29.123603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:51:29.124513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:51:29.125459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:51:29.126274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [parameters: {'id': 'bf3f7e897cfa4c0c92305e0c54fe20cb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:29.127160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:29.130480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf3f7e897cfa4c0c92305e0c54fe20cb. Oct 17 06:51:29.132885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.135191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 370/747] 10.222.0.36 () {60 vars in 1140 bytes} [Mon Oct 17 06:51:28 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:29.136472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.142044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:29.169926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.170756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.171182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.171788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.172618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.174250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:29.184446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:29.185499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:29.186684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:29.189462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 378/748] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:29.195505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.238210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:29.261810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb6e0-7c13-4987-bad8-9b4871c901fa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.300520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.301537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.302045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.302632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.311606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.313630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:29.326270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:29.327486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:29.328410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:29.329246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2b54a-2683-4526-a660-4e9e91140d31 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.335337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 371/749] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:29.337446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.373179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.373698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.373978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.374250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.374881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:29.375190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.376861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.387282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 379/750] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:29.389460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.412439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:29.452331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f527e8b-373c-433a-add7-17e11f237f54 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.489171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d50101-3bfb-4fd1-80e3-2f2e989706f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.489772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d50101-3bfb-4fd1-80e3-2f2e989706f2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.490046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d50101-3bfb-4fd1-80e3-2f2e989706f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.490313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d50101-3bfb-4fd1-80e3-2f2e989706f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.491563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d50101-3bfb-4fd1-80e3-2f2e989706f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:29.491864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d50101-3bfb-4fd1-80e3-2f2e989706f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.493295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d50101-3bfb-4fd1-80e3-2f2e989706f2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.498095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:29.504872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b979a-70e1-431d-ac2b-0101e69fdc40 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.518363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 372/751] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:29.518363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1f2f9-52c8-49f4-aa7b-f0cdfefd269a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.518363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1f2f9-52c8-49f4-aa7b-f0cdfefd269a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.518363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1f2f9-52c8-49f4-aa7b-f0cdfefd269a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.518363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 373/752] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:29.522801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bc7a0-ee47-4333-adbc-c17de0b79199 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.523080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bc7a0-ee47-4333-adbc-c17de0b79199 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.523391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bc7a0-ee47-4333-adbc-c17de0b79199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.543696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5a7bc7a0-ee47-4333-adbc-c17de0b79199 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:29.545394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5a7bc7a0-ee47-4333-adbc-c17de0b79199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:29.555685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.556801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.557257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.557712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.558618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:29.559141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.561060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.568310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 380/753] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:29.574723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.631584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 374/754] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:29 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:29.699052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dc66c-6da5-4569-9ced-6c3b63efc72e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.738838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.739710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.740139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.740616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.741494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] RBAC: Adding query filter params (interface=admin, service_id=f2828796c3c8462f88bfd67e75d451a7, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:29.741804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.743163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.749057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 381/755] 10.222.0.36 () {56 vars in 1249 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/endpoints?service_id=f2828796c3c8462f88bfd67e75d451a7&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:29.756823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bc7a0-ee47-4333-adbc-c17de0b79199 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.820560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:29.864117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bc7a0-ee47-4333-adbc-c17de0b79199 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:29.900212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:29.901154 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:29.901616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:29.902086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:29.902876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:29.904458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:29.911946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 375/756] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:51:29.935585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:29.967074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:30.055956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ed85b-3d1e-4a51-82c4-40fa2341d46e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:30.095001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:30.095756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.096099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.096437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] PATH_INFO: `/v3/services/image` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.097087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:30.098524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:30.107694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:30.108862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:30.109883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:30.111255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 382/757] 10.222.0.36 () {56 vars in 1105 bytes} [Mon Oct 17 06:51:29 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:30.117050 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:30.225140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47c82f-9aa7-419f-a324-876e79f8b26d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:30.245681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:30.265377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9a85f-c60c-4acb-a288-8985035f1013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:30.266037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9a85f-c60c-4acb-a288-8985035f1013 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.266310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9a85f-c60c-4acb-a288-8985035f1013 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.266574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9a85f-c60c-4acb-a288-8985035f1013 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.267253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9a85f-c60c-4acb-a288-8985035f1013 None admin] RBAC: Adding query filter params (name=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:30.267558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9a85f-c60c-4acb-a288-8985035f1013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:30.268928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9a85f-c60c-4acb-a288-8985035f1013 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:30.274876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 376/758] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:30.280298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:30.345369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1179a76-e152-4df8-9e9f-0c4ca92704b6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.345556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1179a76-e152-4df8-9e9f-0c4ca92704b6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.345831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1179a76-e152-4df8-9e9f-0c4ca92704b6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.347430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 377/759] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:30.354749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190840-943c-4c3a-bdc0-80517754f44a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.355031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190840-943c-4c3a-bdc0-80517754f44a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.355337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190840-943c-4c3a-bdc0-80517754f44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.374627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-21190840-943c-4c3a-bdc0-80517754f44a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:30.375831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-21190840-943c-4c3a-bdc0-80517754f44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:30.377883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:30.394181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07529c9-8b0c-4e2b-afbe-7f14de0369bd None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:30.435093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:30.435718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.435995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.436517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.437140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] RBAC: Adding query filter params (type=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:30.437435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:30.438796 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:30.449470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 383/760] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:30.451014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:30.471377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 378/761] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:30 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:30.562971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300ed3f-10ac-44eb-99e8-d7e165ceb414 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:30.599308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:30.600479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.600754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.601017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.601780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] RBAC: Adding query filter params (interface=public, service_id=16939f5d71464d1892bc9279fdd4afc3, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:30.602078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:30.603528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:30.609636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 384/762] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:51:30 2022] GET /identity/v3/endpoints?service_id=16939f5d71464d1892bc9279fdd4afc3&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:30.615362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190840-943c-4c3a-bdc0-80517754f44a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:30.714697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190840-943c-4c3a-bdc0-80517754f44a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:30.728784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:30.750813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:30.751456 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.751730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.751996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.752586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:30.754712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:30.757450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:30.769000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 379/763] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:51:30 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 06:51:30.779597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:30.874983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8167182-37a0-470e-9f77-a3c717c8d61e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:30.915600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:30.916680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:30.917170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:30.917647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:30.918550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:30.920210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:30.930384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:30.931502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:51:30.932348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:30.939345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 385/764] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:30.942947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:31.049799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52685c47-52c3-41cb-8dac-f5f7b19f4b96 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:31.086104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:31.087075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:31.087579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:31.088049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:31.088882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:31.089383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:31.091066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:31.096653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 380/765] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:31.103063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:31.196512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:31.216754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dffe4-c63d-49dc-8f87-15300b7b9927 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:31.251337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459b1a9-bc00-4db0-ba57-c1fe9574fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:31.252332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459b1a9-bc00-4db0-ba57-c1fe9574fb2b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:31.252914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459b1a9-bc00-4db0-ba57-c1fe9574fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:31.253392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459b1a9-bc00-4db0-ba57-c1fe9574fb2b None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:31.254204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459b1a9-bc00-4db0-ba57-c1fe9574fb2b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:31.254722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459b1a9-bc00-4db0-ba57-c1fe9574fb2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:31.256822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459b1a9-bc00-4db0-ba57-c1fe9574fb2b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:31.262566 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 386/766] 10.222.0.36 () {56 vars in 1112 bytes} [Mon Oct 17 06:51:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:31.268560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:31.313948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:31.377323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e8a3b-9a11-4790-b48a-6c7c713a3631 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:31.412273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62e7de-b801-4937-838a-65430df8a86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:31.413229 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e7de-b801-4937-838a-65430df8a86e None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:31.413769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e7de-b801-4937-838a-65430df8a86e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:31.414238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e7de-b801-4937-838a-65430df8a86e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:31.415163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62e7de-b801-4937-838a-65430df8a86e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:31.416746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62e7de-b801-4937-838a-65430df8a86e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:31.438976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 381/767] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:31.903763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe351a-9d27-4114-bc02-d68f0b55ca1c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:31.904432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe351a-9d27-4114-bc02-d68f0b55ca1c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:31.904936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe351a-9d27-4114-bc02-d68f0b55ca1c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:31.906743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 387/768] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:31.911702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bb2ce-adb2-4c1c-b19b-83c8ab9fef1d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:31.912671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bb2ce-adb2-4c1c-b19b-83c8ab9fef1d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:31.912754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bb2ce-adb2-4c1c-b19b-83c8ab9fef1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:31.931657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-347bb2ce-adb2-4c1c-b19b-83c8ab9fef1d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:31.932825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-347bb2ce-adb2-4c1c-b19b-83c8ab9fef1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:32.007758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 382/769] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:31 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:32.201661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b8747-e0ba-4d8e-8d3f-0eb1776e8821 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:32.202169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b8747-e0ba-4d8e-8d3f-0eb1776e8821 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:32.202279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b8747-e0ba-4d8e-8d3f-0eb1776e8821 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:32.203795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 388/770] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:32.209733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae81d7-8d74-4c11-b23f-29416f038812 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:32.210016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae81d7-8d74-4c11-b23f-29416f038812 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:32.210285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae81d7-8d74-4c11-b23f-29416f038812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:32.228205 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2aae81d7-8d74-4c11-b23f-29416f038812 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:32.229375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2aae81d7-8d74-4c11-b23f-29416f038812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:32.246490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:32.286380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b8747-e0ba-4d8e-8d3f-0eb1776e8821 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:32.304966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 383/771] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:32 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:32.397406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b8747-e0ba-4d8e-8d3f-0eb1776e8821 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:32.433271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:32.434194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:32.434649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:32.435092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:32.435968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:32.437944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:32.440865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING keystone.server.flask.common [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 06:51:32.455291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.sql.core [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 06:51:32.455291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 06:51:32.455291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [parameters: {'user_id': 'ef687af7d55d4aa28c2aefecedbbc45a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 06:51:32.455291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81487) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Oct 17 06:51:32.472700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:51:32.473573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application yield resource Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise self.value Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application session.commit() Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:51:32.474537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:51:32.475972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:51:32.477177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:51:32.478393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ef687af7d55d4aa28c2aefecedbbc45a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:32.479359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 17 06:51:32.480442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 17 06:51:32.481421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 17 06:51:32.481421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 17 06:51:32.481421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:51:32.481421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:51:32.481421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 06:51:32.481421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.487535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 389/772] 10.222.0.36 () {60 vars in 1131 bytes} [Mon Oct 17 06:51:32 2022] POST /identity/v3/users => generated 160 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:32.492364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae81d7-8d74-4c11-b23f-29416f038812 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:32.581025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:32.591540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:32.603401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae81d7-8d74-4c11-b23f-29416f038812 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:32.647313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:32.648010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:32.648292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:32.648560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:32.657995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:32.659990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:32.674661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:32.675672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:32.676618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 384/773] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:51:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:32.681773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:32.699274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:32.705338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709956a-8185-4fd9-957e-bbfefc4e5e5b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:32.743045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:32.744109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:32.744656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:32.745125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:32.745892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:32.747874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:32.758276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:32.759331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:51:32.760191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:32.767786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 390/774] 10.222.0.36 () {56 vars in 1126 bytes} [Mon Oct 17 06:51:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:32.768972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:32.801144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1890ad6-3446-48da-9164-677c09d1fdcb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:32.840141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:32.840735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:32.841016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:32.841285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:32.842008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:32.842269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:32.843844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:32.852187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:32.883889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 385/775] 10.222.0.36 () {56 vars in 1101 bytes} [Mon Oct 17 06:51:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:32.886501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b11d4-2101-4752-a191-f6aae1eb3442 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:32.937904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d01f1-548a-433b-af1c-e85498b64dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:32.938563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d01f1-548a-433b-af1c-e85498b64dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:32.938837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d01f1-548a-433b-af1c-e85498b64dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:32.939107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d01f1-548a-433b-af1c-e85498b64dd8 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:32.939842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d01f1-548a-433b-af1c-e85498b64dd8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:32.940151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d01f1-548a-433b-af1c-e85498b64dd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:32.941587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d01f1-548a-433b-af1c-e85498b64dd8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:32.951364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 391/776] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:32.956716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:33.041526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329c81-4a3f-4f61-b5fd-9f6e81f35ddb None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.041855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329c81-4a3f-4f61-b5fd-9f6e81f35ddb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.042074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329c81-4a3f-4f61-b5fd-9f6e81f35ddb None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.043126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:33.044892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 392/777] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:33.049347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dcf63-7eba-4c87-b6a7-49a77e47cdc5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.051393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dcf63-7eba-4c87-b6a7-49a77e47cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.051393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dcf63-7eba-4c87-b6a7-49a77e47cdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.070778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cd5dcf63-7eba-4c87-b6a7-49a77e47cdc5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:33.072402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cd5dcf63-7eba-4c87-b6a7-49a77e47cdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:33.078638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863a6ab-617e-4de5-a41d-8921bcf4d819 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:33.120656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:33.121292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.121572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.121844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.122468 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:33.122773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:33.124184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:33.132513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 386/778] 10.222.0.36 () {56 vars in 1122 bytes} [Mon Oct 17 06:51:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:33.137600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:33.167235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 393/779] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:33 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:33.255409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cfe2f-d1fd-4a63-b09d-d1b8b471bb1e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:33.290057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:33.290656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.290928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.291193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.291902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:33.293263 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:33.297884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:33.312826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 387/780] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:33.436972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5dcf63-7eba-4c87-b6a7-49a77e47cdc5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:33.504879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:33.545630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5dcf63-7eba-4c87-b6a7-49a77e47cdc5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:33.587393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:33.588178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.588547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.588907 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.589564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:33.591006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:33.599631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:33.600826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:51:33.601963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:33.607253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 394/781] 10.222.0.36 () {56 vars in 1105 bytes} [Mon Oct 17 06:51:33 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:33.609613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:33.715075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd39f71-2411-431b-a2b1-f0f806d5c492 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:33.748739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbd81d-73ca-4ca4-899a-338bd3ee010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:33.749485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbd81d-73ca-4ca4-899a-338bd3ee010e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.749856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbd81d-73ca-4ca4-899a-338bd3ee010e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.750220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbd81d-73ca-4ca4-899a-338bd3ee010e None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.750930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbd81d-73ca-4ca4-899a-338bd3ee010e None admin] RBAC: Adding query filter params (name=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:33.751352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbd81d-73ca-4ca4-899a-338bd3ee010e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:33.752818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbd81d-73ca-4ca4-899a-338bd3ee010e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:33.762871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 388/782] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:33.764370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:33.793226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3337e-2571-43dd-a78e-324e437cdaab None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.793829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3337e-2571-43dd-a78e-324e437cdaab None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.794107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3337e-2571-43dd-a78e-324e437cdaab None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.795901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 389/783] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:33.801503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc9a69-b02b-49f5-82b9-e06ee35a14a2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.801862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc9a69-b02b-49f5-82b9-e06ee35a14a2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.802194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc9a69-b02b-49f5-82b9-e06ee35a14a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.823416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c1bc9a69-b02b-49f5-82b9-e06ee35a14a2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:33.824720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c1bc9a69-b02b-49f5-82b9-e06ee35a14a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:33.874877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268ce6-8659-4522-a514-f9477eee39a6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:33.879288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:33.894858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 390/784] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:33.903007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:33.915225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:33.915836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:33.916109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:33.916374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:33.916983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] RBAC: Adding query filter params (type=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:33.917279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:33.918626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:33.928817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 395/785] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:33.930287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc9a69-b02b-49f5-82b9-e06ee35a14a2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:34.029759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc9a69-b02b-49f5-82b9-e06ee35a14a2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:34.065048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:34.065569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:34.065843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:34.066112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:34.066764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:34.068156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:34.093620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 391/786] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:51:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:34.161622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:34.272945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d27cae-bb13-4750-b484-e166028fc232 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:34.313731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:34.314045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:34.314230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:34.315177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:34.327695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:34.332474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:34.333377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:34.334782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 396/787] 10.222.0.36 () {56 vars in 1120 bytes} [Mon Oct 17 06:51:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:34.341133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:34.504578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:34.562708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee767d3-50a8-4ea7-bd73-cb2c86e84713 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:34.596902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:34.597425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:34.597701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:34.597971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:34.598622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:34.598921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:34.600868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:34.609499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 392/788] 10.222.0.36 () {56 vars in 1115 bytes} [Mon Oct 17 06:51:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:34.614774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:34.661196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:34.717227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72482718-2e3a-4097-bfb9-4703e3edd12b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:34.748874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:34.749483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:34.749756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:34.750024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:34.760753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:34.762195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:34.775720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:34.776683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:34.777580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:34.777580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:34.777580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:34.779342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 397/789] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:34.782508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:34.884679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95999-2ffb-4166-82e8-840ed1112cf6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:34.891688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:34.920342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:34.920951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:34.921225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:34.921491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:34.922120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:34.922417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:34.923801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:34.939251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 393/790] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:34.941536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.038376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:35.045393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2deb7e7-b8c7-4543-b4e7-957a6f4efc0a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:35.078804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:35.079444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.079720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.079985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.087792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:35.089209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:35.100456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:35.101396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:51:35.102336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:35.106324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 398/791] 10.222.0.36 () {56 vars in 1093 bytes} [Mon Oct 17 06:51:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:35.107336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.211775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b927f6a-ea16-49d7-956e-deb520a5eb11 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:35.245613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:35.246212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.246483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.246744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.247470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:35.247768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:35.249177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:35.266605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 394/792] 10.222.0.36 () {56 vars in 1133 bytes} [Mon Oct 17 06:51:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:35.272646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.371005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef675b5-7c72-408c-8ac6-4c8771716425 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:35.407743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc253749-f1e0-40e5-8c70-298705234d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:35.408292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc253749-f1e0-40e5-8c70-298705234d29 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.408564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc253749-f1e0-40e5-8c70-298705234d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.408831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc253749-f1e0-40e5-8c70-298705234d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.419986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc253749-f1e0-40e5-8c70-298705234d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:35.421416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc253749-f1e0-40e5-8c70-298705234d29 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fc253749-f1e0-40e5-8c70-298705234d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:35.435110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:35.436385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:35.437304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:35.437304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:35.437304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:35.439292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 399/793] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:35.442228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.467467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:35.491963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565597e-c3b9-40a7-ace8-3077f00fa5da None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.492328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565597e-c3b9-40a7-ace8-3077f00fa5da None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.492670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565597e-c3b9-40a7-ace8-3077f00fa5da None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.494204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 400/794] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:51:35.498989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de76a57-e43a-4d45-9578-808c94386e93 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.499384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de76a57-e43a-4d45-9578-808c94386e93 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.499777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de76a57-e43a-4d45-9578-808c94386e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.520096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9de76a57-e43a-4d45-9578-808c94386e93 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:35.521326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9de76a57-e43a-4d45-9578-808c94386e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:35.523283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:35.551707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face9b4e-31b8-4264-840f-3d21a0aada6e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:35.589879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:35.590777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.591290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.591449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 401/795] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:35.591895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.592529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:35.592826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:35.594185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:35.610104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 395/796] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:35.616118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de76a57-e43a-4d45-9578-808c94386e93 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.713110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de76a57-e43a-4d45-9578-808c94386e93 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:35.746287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:35.746902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.747175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.747475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.753864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:35.755336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:35.757024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:35.768630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:35.769574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:35.770472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:35.774581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 402/797] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:35.775811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.859609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.865067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:35.882044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4df9a-4cb3-4277-8574-99910c2d9068 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:35.916352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:35.917382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:35.917870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:35.918292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:35.919169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:35.919697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:35.921388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:35.939986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 396/798] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:35.941390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:35.964673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53cb0-e706-4e56-9788-432efee175b8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:36.004831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:36.005435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:36.005761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:36.006027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:36.007100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:36.008558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:36.010105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 403/799] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:36.015701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:36.045383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cd5ad-36b6-4196-923e-9b538abb771b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:36.080191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38738943-5987-41a4-bd17-961c450e6d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:36.080837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38738943-5987-41a4-bd17-961c450e6d0e None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:36.081107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38738943-5987-41a4-bd17-961c450e6d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:36.081372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38738943-5987-41a4-bd17-961c450e6d0e None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/74a91af405e24a0b8e9c893e1c299db4/roles/c6bee23227284e0e907145bc459b5db7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:36.083504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38738943-5987-41a4-bd17-961c450e6d0e None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=74a91af405e24a0b8e9c893e1c299db4, role_id=c6bee23227284e0e907145bc459b5db7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:36.085495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38738943-5987-41a4-bd17-961c450e6d0e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:36.086184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:36.099833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 397/800] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:51:35 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/74a91af405e24a0b8e9c893e1c299db4/roles/c6bee23227284e0e907145bc459b5db7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:51:36.124327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce9ed-ae29-4adc-9d02-b4d2403df587 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:36.170697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00d7cd-ca12-4e9d-8ecb-ae81405ab453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:36.171948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00d7cd-ca12-4e9d-8ecb-ae81405ab453 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:36.172526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00d7cd-ca12-4e9d-8ecb-ae81405ab453 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:36.173120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00d7cd-ca12-4e9d-8ecb-ae81405ab453 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:36.174479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00d7cd-ca12-4e9d-8ecb-ae81405ab453 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:36.177634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00d7cd-ca12-4e9d-8ecb-ae81405ab453 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:36.212224 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 404/801] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:36 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:51:36.300076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eabcb-2389-4401-a607-e71975c85afe None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:36.300561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eabcb-2389-4401-a607-e71975c85afe None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:36.300561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eabcb-2389-4401-a607-e71975c85afe None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:36.301881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 398/802] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:36.308425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32895596-a7d1-42c3-ba32-f31298e5ad9c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:36.308714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32895596-a7d1-42c3-ba32-f31298e5ad9c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:36.308992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32895596-a7d1-42c3-ba32-f31298e5ad9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:36.328725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-32895596-a7d1-42c3-ba32-f31298e5ad9c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:36.329926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-32895596-a7d1-42c3-ba32-f31298e5ad9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:36.408878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 405/803] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:36.675222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9eabcb-2389-4401-a607-e71975c85afe None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:36.679651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:36.781957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9eabcb-2389-4401-a607-e71975c85afe None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:36.817471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:36.818106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:36.818912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:36.819184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:36.819873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:36.821237 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:36.828938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.sql.core [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:51:36.828938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:51:36.828938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [parameters: {'id': '08cb631c50e94b7190080007bbf65e00', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 06:51:36.828938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81488) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:51:36.836587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 06:51:36.837383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application yield resource Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise self.value Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application session.commit() Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:51:36.838360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:51:36.839754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:51:36.840663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:51:36.841420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [parameters: {'id': '08cb631c50e94b7190080007bbf65e00', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 06:51:36.842281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:36.843049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:36.843976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 399/804] 10.222.0.36 () {60 vars in 1137 bytes} [Mon Oct 17 06:51:36 2022] POST /identity/v3/domains => generated 190 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:36.844671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32895596-a7d1-42c3-ba32-f31298e5ad9c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:36.892954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:36.946378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32895596-a7d1-42c3-ba32-f31298e5ad9c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:36.980275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea430f-5456-4d9f-a6b7-7fcbaf1f04b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:36.980933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea430f-5456-4d9f-a6b7-7fcbaf1f04b1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:36.981209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea430f-5456-4d9f-a6b7-7fcbaf1f04b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:36.981478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea430f-5456-4d9f-a6b7-7fcbaf1f04b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:36.993149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea430f-5456-4d9f-a6b7-7fcbaf1f04b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:36.994636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea430f-5456-4d9f-a6b7-7fcbaf1f04b1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-6cea430f-5456-4d9f-a6b7-7fcbaf1f04b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:37.010256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:37.011520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:37.012467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:37.012467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:37.012467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:37.012467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 406/805] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:37.017235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:37.086430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:37.122090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cfc52-1636-492e-bf89-4b488e8c12a8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:37.166533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49f6c1-4d54-4ff8-9aa0-616e8de0789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:37.167135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f6c1-4d54-4ff8-9aa0-616e8de0789f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:37.167439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f6c1-4d54-4ff8-9aa0-616e8de0789f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:37.167707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f6c1-4d54-4ff8-9aa0-616e8de0789f None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:37.168344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49f6c1-4d54-4ff8-9aa0-616e8de0789f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:37.168641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49f6c1-4d54-4ff8-9aa0-616e8de0789f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:37.170019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49f6c1-4d54-4ff8-9aa0-616e8de0789f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:37.181722 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 400/806] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:38.257362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c82102-a09d-458f-bbe6-c31e4e6f2851 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:38.257894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c82102-a09d-458f-bbe6-c31e4e6f2851 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:38.257894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c82102-a09d-458f-bbe6-c31e4e6f2851 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:38.259241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 407/807] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:38.264587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88527a-0635-4074-aa29-cd887fd0898e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:38.264963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88527a-0635-4074-aa29-cd887fd0898e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:38.265125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88527a-0635-4074-aa29-cd887fd0898e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:38.282924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bb88527a-0635-4074-aa29-cd887fd0898e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:38.284101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb88527a-0635-4074-aa29-cd887fd0898e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:38.354013 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 401/808] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:38.420165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6d203-83c3-4203-91e9-662b66c754dd None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:38.420356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6d203-83c3-4203-91e9-662b66c754dd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:38.420624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6d203-83c3-4203-91e9-662b66c754dd None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:38.421931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 408/809] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:38.427059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fe71a-a27d-40d2-a4b4-b2e4444960e2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:38.427430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fe71a-a27d-40d2-a4b4-b2e4444960e2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:38.427768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fe71a-a27d-40d2-a4b4-b2e4444960e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:38.446204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5a5fe71a-a27d-40d2-a4b4-b2e4444960e2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:38.447402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5a5fe71a-a27d-40d2-a4b4-b2e4444960e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:38.497469 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:38.511407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 402/810] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:38.624332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6d203-83c3-4203-91e9-662b66c754dd None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:38.720185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:38.724180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6d203-83c3-4203-91e9-662b66c754dd None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:38.756251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:38.756856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:38.757127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:38.757393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:38.767453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:38.772582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:38.773641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:38.774509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 409/811] 10.222.0.36 () {56 vars in 1120 bytes} [Mon Oct 17 06:51:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:38.778952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fe71a-a27d-40d2-a4b4-b2e4444960e2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:38.779505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:38.876544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fe71a-a27d-40d2-a4b4-b2e4444960e2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:38.880181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32f178-d05b-4bdf-ba49-38df52437d0f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:38.909268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:38.909884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:38.910158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:38.910426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:38.911514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:38.912540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:38.912967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:38.913127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:38.913404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:38.913671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:38.914312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:38.914509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 403/812] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:38.914596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:38.915998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:38.920082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:38.923928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 410/813] 10.222.0.36 () {56 vars in 1115 bytes} [Mon Oct 17 06:51:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:38.929973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:38.953222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:38.974113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:39.017692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdc09e-1b84-4e18-a106-b8b740e0d8a7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:39.026684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b660738-99e5-4bdb-9440-c40df7c5687f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:39.049149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:39.049746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:39.050018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:39.050281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:39.050907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:39.052829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:39.058927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:39.059602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:39.059875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:39.060142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:39.071708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:39.073126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:39.087242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 404/814] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:51:38 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:39.088243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:39.089337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:39.090259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:39.090259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:39.090259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:39.090259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 411/815] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:51:39.095596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:39.208529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fcfb4-d108-4ce4-8252-784ca6dd9905 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:39.222667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:39.245772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:39.246292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:39.246563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:39.246824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:39.247482 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:39.247780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:39.249137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:39.262334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 405/816] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:39.268062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:39.341999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:39.367049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed9112-6221-44c2-8a68-df0ef376097f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:39.399000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:39.399646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:39.399924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:39.400191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:39.407385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:39.408820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:51:39.419952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:39.421011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:39.422000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 412/817] 10.222.0.36 () {56 vars in 1093 bytes} [Mon Oct 17 06:51:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:39.426829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:39.523031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774f777-9230-4c01-8de3-80929a6c74b7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:39.557298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:39.557891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:39.558165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:39.558430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:39.559138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:39.559478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:39.561167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:39.577916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 406/818] 10.222.0.36 () {56 vars in 1133 bytes} [Mon Oct 17 06:51:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:39.584226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:39.680576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7f5c4-422a-467b-8167-f257968122d1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:39.712817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:39.713402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:39.713678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:39.713948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:39.716477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:39.782030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:39.783427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:39.878595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 413/819] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:51:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:51:39.884315 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:39.914084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:39.982988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384e705-31e8-4298-9cb4-57c2a4f04919 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:40.016027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:40.016633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:40.016904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:40.017170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:40.027340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:40.028763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:51:40.042839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:40.043785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:51:40.044690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:51:40.044690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:51:40.044690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:40.044690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 407/820] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:40.049704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:40.082199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:40.145575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c8bf-30c6-4fb6-a1d0-4b32fec11060 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:40.178965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050db39a-522f-4228-9c49-177f458229fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:40.179587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050db39a-522f-4228-9c49-177f458229fb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:40.179862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050db39a-522f-4228-9c49-177f458229fb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:40.180127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050db39a-522f-4228-9c49-177f458229fb None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:40.180750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050db39a-522f-4228-9c49-177f458229fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:40.181048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050db39a-522f-4228-9c49-177f458229fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:40.182394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050db39a-522f-4228-9c49-177f458229fb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:40.193702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 414/821] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:40.199364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:40.277834 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:40.296765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae894530-3417-4a20-9ed3-d204d0838e85 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:40.329776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:40.330397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:40.330675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:40.330949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:40.337961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:40.339431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:51:40.350897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:40.351895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:40.353211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 408/822] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:51:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:40.357858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050db39a-522f-4228-9c49-177f458229fb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:40.447771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:40.454822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050db39a-522f-4228-9c49-177f458229fb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:40.487566 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40349f41-22fd-46aa-a839-d3c1b7903c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:40.488161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40349f41-22fd-46aa-a839-d3c1b7903c05 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:40.488439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40349f41-22fd-46aa-a839-d3c1b7903c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:40.488710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40349f41-22fd-46aa-a839-d3c1b7903c05 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:40.489391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40349f41-22fd-46aa-a839-d3c1b7903c05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:40.489692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40349f41-22fd-46aa-a839-d3c1b7903c05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:40.491738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40349f41-22fd-46aa-a839-d3c1b7903c05 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:40.503542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 415/823] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:51:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:40.509835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:40.605724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43564-a94b-41f9-9810-84e32228d4da None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:40.638384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a5f0d-c27e-4bdc-90ca-8932cb403b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:40.638976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a5f0d-c27e-4bdc-90ca-8932cb403b0e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:40.639287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a5f0d-c27e-4bdc-90ca-8932cb403b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:40.639559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a5f0d-c27e-4bdc-90ca-8932cb403b0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:40.640270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a5f0d-c27e-4bdc-90ca-8932cb403b0e None admin] RBAC: Adding query filter params (role.id=c6bee23227284e0e907145bc459b5db7, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=74a91af405e24a0b8e9c893e1c299db4) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:40.640571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a5f0d-c27e-4bdc-90ca-8932cb403b0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:40.641993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a5f0d-c27e-4bdc-90ca-8932cb403b0e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:40.647564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 409/824] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:51:40 2022] GET /identity/v3/role_assignments?user.id=74a91af405e24a0b8e9c893e1c299db4&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=c6bee23227284e0e907145bc459b5db7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:41.233984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cf44-5c67-4048-b132-8d2078431517 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:41.234261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cf44-5c67-4048-b132-8d2078431517 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:41.234532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cf44-5c67-4048-b132-8d2078431517 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:41.235905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 416/825] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:41.241161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac4916-6e8f-479c-8072-ecdaf2087f52 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:41.241444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac4916-6e8f-479c-8072-ecdaf2087f52 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:41.241708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac4916-6e8f-479c-8072-ecdaf2087f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:41.254623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:41.260347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-52ac4916-6e8f-479c-8072-ecdaf2087f52 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:41.261524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-52ac4916-6e8f-479c-8072-ecdaf2087f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:41.327726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 410/826] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:41.594124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39cf44-5c67-4048-b132-8d2078431517 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:41.690366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39cf44-5c67-4048-b132-8d2078431517 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:41.722625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:41.723242 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:41.723519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:41.723784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:41.733287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:41.737887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:41.738955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:51:41.740334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 417/827] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:41.744810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac4916-6e8f-479c-8072-ecdaf2087f52 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:41.838955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac4916-6e8f-479c-8072-ecdaf2087f52 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:41.871051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:41.871625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:41.871893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:41.872158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:41.872790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:41.873087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:41.874432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:41.876512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:41.883501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 411/828] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:51:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:41.888912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:41.904393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:41.983102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fd7c3-67ea-4607-8a29-9814ed2d9112 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:42.016028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:42.016620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:42.016888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:42.017149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] PATH_INFO: `/v3/users/afd279752c2d4a29b58b64a2368db98c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:42.029665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] RBAC: Authorizing `identity:get_user(user_id=afd279752c2d4a29b58b64a2368db98c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:42.031078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:42.032638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 418/829] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:51:41 2022] GET /identity/v3/users/afd279752c2d4a29b58b64a2368db98c => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:42.038223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:42.133961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8db3b-2ebf-430f-88e3-49bab5a681b6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:42.166332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106f331-9179-4d04-aef9-26320a2710d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:42.166944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106f331-9179-4d04-aef9-26320a2710d9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:42.167230 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106f331-9179-4d04-aef9-26320a2710d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:42.167504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106f331-9179-4d04-aef9-26320a2710d9 None admin] PATH_INFO: `/v3/projects/957994c9dfaa49c291bbffc0c7569431` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:42.168580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106f331-9179-4d04-aef9-26320a2710d9 None admin] RBAC: Authorizing `identity:get_project(project_id=957994c9dfaa49c291bbffc0c7569431)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:42.169994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106f331-9179-4d04-aef9-26320a2710d9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:42.171624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 412/830] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:51:42 2022] GET /identity/v3/projects/957994c9dfaa49c291bbffc0c7569431 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:42.178059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:42.244686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:42.272741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8b0ef-ad54-4d18-8f0b-95ccd4285d67 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:42.305689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f4a4d-94da-4853-8bf1-1cd4a7d8e33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:42.306333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f4a4d-94da-4853-8bf1-1cd4a7d8e33c None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:42.306611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f4a4d-94da-4853-8bf1-1cd4a7d8e33c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:42.306880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f4a4d-94da-4853-8bf1-1cd4a7d8e33c None admin] PATH_INFO: `/v3/projects/957994c9dfaa49c291bbffc0c7569431/users/afd279752c2d4a29b58b64a2368db98c/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:42.308771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f4a4d-94da-4853-8bf1-1cd4a7d8e33c None admin] RBAC: Authorizing `identity:create_grant(project_id=957994c9dfaa49c291bbffc0c7569431, user_id=afd279752c2d4a29b58b64a2368db98c, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:42.310198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f4a4d-94da-4853-8bf1-1cd4a7d8e33c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:42.322385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 419/831] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:51:42 2022] PUT /identity/v3/projects/957994c9dfaa49c291bbffc0c7569431/users/afd279752c2d4a29b58b64a2368db98c/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:51:44.407251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22686d4-024b-4687-af4f-f1274e864315 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:44.407790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22686d4-024b-4687-af4f-f1274e864315 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:44.407790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22686d4-024b-4687-af4f-f1274e864315 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:44.409025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 413/832] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:44.414952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16ca7-6470-4a2c-9383-aecb1c6c1871 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:44.415399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16ca7-6470-4a2c-9383-aecb1c6c1871 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:44.415495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16ca7-6470-4a2c-9383-aecb1c6c1871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:44.433329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-76e16ca7-6470-4a2c-9383-aecb1c6c1871 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:44.434516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-76e16ca7-6470-4a2c-9383-aecb1c6c1871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:44.497741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 420/833] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:44.759795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22686d4-024b-4687-af4f-f1274e864315 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:44.824440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:44.859420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22686d4-024b-4687-af4f-f1274e864315 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:44.890667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:44.891291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:44.891564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:44.891830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:44.901903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:51:44.906500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:51:44.907693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:51:44.908769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 414/834] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:51:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:51:44.914076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16ca7-6470-4a2c-9383-aecb1c6c1871 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:44.971671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:45.008609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16ca7-6470-4a2c-9383-aecb1c6c1871 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:45.041179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:45.041708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:45.041981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:45.042244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:45.042882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:45.043174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:45.044538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:45.052120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 421/835] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:51:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:45.057831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:45.154575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb35f-f9c4-4d08-8bc4-ede7736933c7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:45.179659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:45.188626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:45.189232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:45.189503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:45.189769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] PATH_INFO: `/v3/users/afd279752c2d4a29b58b64a2368db98c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:45.190847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] RBAC: Authorizing `identity:get_user(user_id=afd279752c2d4a29b58b64a2368db98c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:45.192283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:45.194339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 415/836] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:51:45 2022] GET /identity/v3/users/afd279752c2d4a29b58b64a2368db98c => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:45.200060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:45.294364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21644c28-0b0d-4b63-adfc-3218367114fb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:45.316049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:45.326842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:45.327448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:45.327768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:45.328034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:45.390571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:45.392019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:45.484398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 422/837] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:51:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:51:45.490317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:45.584828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77344f-13f3-46e3-ada6-a7814fb26038 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:45.616651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3445462-9e3f-420e-b3bb-134269468387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:45.617250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3445462-9e3f-420e-b3bb-134269468387 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:45.617519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3445462-9e3f-420e-b3bb-134269468387 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:45.617782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3445462-9e3f-420e-b3bb-134269468387 None admin] PATH_INFO: `/v3/projects/957994c9dfaa49c291bbffc0c7569431` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:45.618851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3445462-9e3f-420e-b3bb-134269468387 None admin] RBAC: Authorizing `identity:get_project(project_id=957994c9dfaa49c291bbffc0c7569431)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:45.620276 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3445462-9e3f-420e-b3bb-134269468387 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:45.621839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 416/838] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:51:45 2022] GET /identity/v3/projects/957994c9dfaa49c291bbffc0c7569431 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:45.627708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:45.651105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:45.833735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07ad2-4941-43a7-9482-4cb6642d8449 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:45.866186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3224b13-df49-45f0-bb5b-8bb039142305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:45.866709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3224b13-df49-45f0-bb5b-8bb039142305 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:45.866981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3224b13-df49-45f0-bb5b-8bb039142305 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:45.867270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3224b13-df49-45f0-bb5b-8bb039142305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:45.867978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3224b13-df49-45f0-bb5b-8bb039142305 None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=957994c9dfaa49c291bbffc0c7569431, user.id=afd279752c2d4a29b58b64a2368db98c) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:45.868280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3224b13-df49-45f0-bb5b-8bb039142305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:45.869674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3224b13-df49-45f0-bb5b-8bb039142305 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:45.875149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 423/839] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:51:45 2022] GET /identity/v3/role_assignments?user.id=afd279752c2d4a29b58b64a2368db98c&scope.project.id=957994c9dfaa49c291bbffc0c7569431&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:47.984759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9469425-c4d7-4a6d-a8ac-91ea3ce6c909 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:47.985309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9469425-c4d7-4a6d-a8ac-91ea3ce6c909 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:47.985309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9469425-c4d7-4a6d-a8ac-91ea3ce6c909 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:47.986607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 417/840] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:47.991894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a302-8ade-40ae-9f26-bc10556ce490 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:47.992327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a302-8ade-40ae-9f26-bc10556ce490 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:47.992432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a302-8ade-40ae-9f26-bc10556ce490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:48.009750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ecb0a302-8ade-40ae-9f26-bc10556ce490 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:48.010908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ecb0a302-8ade-40ae-9f26-bc10556ce490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:48.073811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 424/841] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:48.341143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9469425-c4d7-4a6d-a8ac-91ea3ce6c909 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:48.400850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:48.439139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9469425-c4d7-4a6d-a8ac-91ea3ce6c909 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:48.472354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62357842-4142-4863-a95b-d6d3bc53b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:48.472997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62357842-4142-4863-a95b-d6d3bc53b2de None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:48.473273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62357842-4142-4863-a95b-d6d3bc53b2de None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:48.473542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62357842-4142-4863-a95b-d6d3bc53b2de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:48.474628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62357842-4142-4863-a95b-d6d3bc53b2de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:48.476081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62357842-4142-4863-a95b-d6d3bc53b2de None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:48.477620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 418/842] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:48.483245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a302-8ade-40ae-9f26-bc10556ce490 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:48.500723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:48.577100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a302-8ade-40ae-9f26-bc10556ce490 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:48.608374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341883c-b82a-41bb-9bc6-55f979c5de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:48.608893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341883c-b82a-41bb-9bc6-55f979c5de4d None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:48.609162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341883c-b82a-41bb-9bc6-55f979c5de4d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:48.609423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341883c-b82a-41bb-9bc6-55f979c5de4d None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:48.610051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341883c-b82a-41bb-9bc6-55f979c5de4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:48.611476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341883c-b82a-41bb-9bc6-55f979c5de4d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:48.632313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 425/843] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:51:48 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:51:50.767676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83474164-0089-446b-9e5e-74358a5ff922 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:50.768227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83474164-0089-446b-9e5e-74358a5ff922 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:50.768227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83474164-0089-446b-9e5e-74358a5ff922 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:50.769551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 419/844] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:50.774695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6411632-a338-4d92-bc58-d98f799a081c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:50.775127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6411632-a338-4d92-bc58-d98f799a081c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:50.775248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6411632-a338-4d92-bc58-d98f799a081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:50.793222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c6411632-a338-4d92-bc58-d98f799a081c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:50.794386 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6411632-a338-4d92-bc58-d98f799a081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:50.858500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 426/845] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:51.128429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83474164-0089-446b-9e5e-74358a5ff922 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:51.226887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83474164-0089-446b-9e5e-74358a5ff922 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:51.254040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:51.259990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:51.260603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:51.260879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:51.261146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] PATH_INFO: `/v3/roles/ca9ae3dc2d9b42c0b35eca1fda566496` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:51.262500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] RBAC: Authorizing `identity:get_role(role_id=ca9ae3dc2d9b42c0b35eca1fda566496)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:51.263886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:51.265178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 420/846] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:51:51 2022] GET /identity/v3/roles/ca9ae3dc2d9b42c0b35eca1fda566496 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:51.270966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6411632-a338-4d92-bc58-d98f799a081c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:51.274006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:51.365671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6411632-a338-4d92-bc58-d98f799a081c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:51.397012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:51.397569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:51.397840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:51.398104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] PATH_INFO: `/v3/users/7669a8e5e0274e95a372b9705650819f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:51.410254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] RBAC: Authorizing `identity:get_user(user_id=7669a8e5e0274e95a372b9705650819f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:51.412208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:51.413762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 427/847] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:51:51 2022] GET /identity/v3/users/7669a8e5e0274e95a372b9705650819f => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:51.419507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:51.515683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c1394-1fb0-485c-8dff-70b42ebe2898 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:51.548586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991015f-62d1-4e02-adeb-0085047f8ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:51.549194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991015f-62d1-4e02-adeb-0085047f8ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:51.549473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991015f-62d1-4e02-adeb-0085047f8ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:51.549746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991015f-62d1-4e02-adeb-0085047f8ef4 None admin] PATH_INFO: `/v3/projects/957994c9dfaa49c291bbffc0c7569431` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:51.550837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991015f-62d1-4e02-adeb-0085047f8ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=957994c9dfaa49c291bbffc0c7569431)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:51.552852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991015f-62d1-4e02-adeb-0085047f8ef4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:51.554439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 421/848] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:51:51 2022] GET /identity/v3/projects/957994c9dfaa49c291bbffc0c7569431 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:51.560226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:51.618646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:51.654332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c4b5-3838-4f64-b013-137e5822019e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:51.686273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027d60c-5482-4caa-a53f-089aa0b560ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:51.686898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d60c-5482-4caa-a53f-089aa0b560ae None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:51.687169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d60c-5482-4caa-a53f-089aa0b560ae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:51.687463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d60c-5482-4caa-a53f-089aa0b560ae None admin] PATH_INFO: `/v3/projects/957994c9dfaa49c291bbffc0c7569431/users/7669a8e5e0274e95a372b9705650819f/roles/ca9ae3dc2d9b42c0b35eca1fda566496` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:51.689509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027d60c-5482-4caa-a53f-089aa0b560ae None admin] RBAC: Authorizing `identity:create_grant(project_id=957994c9dfaa49c291bbffc0c7569431, user_id=7669a8e5e0274e95a372b9705650819f, role_id=ca9ae3dc2d9b42c0b35eca1fda566496)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:51.690901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027d60c-5482-4caa-a53f-089aa0b560ae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:51.703039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 428/849] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:51:51 2022] PUT /identity/v3/projects/957994c9dfaa49c291bbffc0c7569431/users/7669a8e5e0274e95a372b9705650819f/roles/ca9ae3dc2d9b42c0b35eca1fda566496 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:51:53.788144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b809-5ba3-4ead-869b-f9f1843befb7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:53.788688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b809-5ba3-4ead-869b-f9f1843befb7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:53.788688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b809-5ba3-4ead-869b-f9f1843befb7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:53.789987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 422/850] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:53.795056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0e89-0940-44a7-8093-39fb5f9168f4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:53.795527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0e89-0940-44a7-8093-39fb5f9168f4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:53.795634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0e89-0940-44a7-8093-39fb5f9168f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:53.813704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c59a0e89-0940-44a7-8093-39fb5f9168f4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:53.814850 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c59a0e89-0940-44a7-8093-39fb5f9168f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:53.877253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 429/851] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:51:54.143664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1b809-5ba3-4ead-869b-f9f1843befb7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:54.209275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:54.243266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1b809-5ba3-4ead-869b-f9f1843befb7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:54.275900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:54.276507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:54.276782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:54.277053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] PATH_INFO: `/v3/roles/ca9ae3dc2d9b42c0b35eca1fda566496` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:54.278405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] RBAC: Authorizing `identity:get_role(role_id=ca9ae3dc2d9b42c0b35eca1fda566496)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:54.279789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:54.281075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 423/852] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:51:54 2022] GET /identity/v3/roles/ca9ae3dc2d9b42c0b35eca1fda566496 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:54.286837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0e89-0940-44a7-8093-39fb5f9168f4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:54.337011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:54.382113 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0e89-0940-44a7-8093-39fb5f9168f4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:54.414353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:54.414969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:54.415262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:54.415534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] PATH_INFO: `/v3/users/7669a8e5e0274e95a372b9705650819f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:54.416616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] RBAC: Authorizing `identity:get_user(user_id=7669a8e5e0274e95a372b9705650819f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:54.418020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:54.419600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 430/853] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:51:54 2022] GET /identity/v3/users/7669a8e5e0274e95a372b9705650819f => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:54.425341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:54.521796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25274a30-5053-458e-a095-564ec4a1f246 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:54.553606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:54.554186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:54.554457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:54.554724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:54.557196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:54.619731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:54.621164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:54.714477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 424/854] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:51:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:51:54.720227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:54.813720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b94354-f591-490f-b44b-0eb370d7d382 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:54.832837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:54.846136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc10ad-7577-4b1a-bcd1-610d47ac2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:54.846738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc10ad-7577-4b1a-bcd1-610d47ac2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:54.847007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc10ad-7577-4b1a-bcd1-610d47ac2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:54.847297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc10ad-7577-4b1a-bcd1-610d47ac2ee9 None admin] PATH_INFO: `/v3/projects/957994c9dfaa49c291bbffc0c7569431` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:54.848391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc10ad-7577-4b1a-bcd1-610d47ac2ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=957994c9dfaa49c291bbffc0c7569431)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:54.849787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc10ad-7577-4b1a-bcd1-610d47ac2ee9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:54.851381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 431/855] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:51:54 2022] GET /identity/v3/projects/957994c9dfaa49c291bbffc0c7569431 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:51:54.857637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:54.890171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:54.954294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ea7ba-e404-42ce-96f8-68dd11ddc474 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:54.986753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7c0f-25e0-41b6-aa40-98996b50803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:54.987307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7c0f-25e0-41b6-aa40-98996b50803d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:54.987585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7c0f-25e0-41b6-aa40-98996b50803d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:54.987855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7c0f-25e0-41b6-aa40-98996b50803d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:54.988561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7c0f-25e0-41b6-aa40-98996b50803d None admin] RBAC: Adding query filter params (role.id=ca9ae3dc2d9b42c0b35eca1fda566496, scope.project.id=957994c9dfaa49c291bbffc0c7569431, user.id=7669a8e5e0274e95a372b9705650819f) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:51:54.988951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7c0f-25e0-41b6-aa40-98996b50803d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:54.990327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7c0f-25e0-41b6-aa40-98996b50803d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:54.995835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 425/856] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:51:54 2022] GET /identity/v3/role_assignments?user.id=7669a8e5e0274e95a372b9705650819f&scope.project.id=957994c9dfaa49c291bbffc0c7569431&role.id=ca9ae3dc2d9b42c0b35eca1fda566496 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:57.093647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573ab57-04d5-4c2b-90f0-a66f72c3000b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:57.094174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573ab57-04d5-4c2b-90f0-a66f72c3000b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:57.094174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573ab57-04d5-4c2b-90f0-a66f72c3000b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:57.095536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 432/857] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:57.101196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed69e0-eaac-4532-8f3d-e741a0f5af44 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:57.101606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed69e0-eaac-4532-8f3d-e741a0f5af44 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:57.101714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed69e0-eaac-4532-8f3d-e741a0f5af44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:57.119097 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-afed69e0-eaac-4532-8f3d-e741a0f5af44 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:57.120308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-afed69e0-eaac-4532-8f3d-e741a0f5af44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:57.184362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 426/858] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:51:57.451235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573ab57-04d5-4c2b-90f0-a66f72c3000b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:57.548172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573ab57-04d5-4c2b-90f0-a66f72c3000b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:57.580506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e905187-07b2-4e39-b032-3d14ac2f611d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:57.581107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e905187-07b2-4e39-b032-3d14ac2f611d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:57.581381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e905187-07b2-4e39-b032-3d14ac2f611d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:57.581648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e905187-07b2-4e39-b032-3d14ac2f611d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:57.583283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e905187-07b2-4e39-b032-3d14ac2f611d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:57.584698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e905187-07b2-4e39-b032-3d14ac2f611d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:57.586215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 433/859] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:51:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:51:57.591961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed69e0-eaac-4532-8f3d-e741a0f5af44 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:51:57.618675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:51:57.687327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed69e0-eaac-4532-8f3d-e741a0f5af44 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:51:57.719255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ff0ad-0140-4a2d-94e8-2a7396ea8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:51:57.719795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ff0ad-0140-4a2d-94e8-2a7396ea8156 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:57.720066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ff0ad-0140-4a2d-94e8-2a7396ea8156 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:57.720331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ff0ad-0140-4a2d-94e8-2a7396ea8156 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:57.720953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ff0ad-0140-4a2d-94e8-2a7396ea8156 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:51:57.722366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ff0ad-0140-4a2d-94e8-2a7396ea8156 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:51:57.747410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 427/860] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:51:57 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:51:59.897887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52afa7-1d05-4be5-ab7f-58c5eeee9217 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:59.898418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52afa7-1d05-4be5-ab7f-58c5eeee9217 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:59.898418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52afa7-1d05-4be5-ab7f-58c5eeee9217 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:59.899783 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 434/861] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:51:59.905041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85e116-8fcf-4f3a-a344-ad5c40dada82 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:51:59.905457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85e116-8fcf-4f3a-a344-ad5c40dada82 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:51:59.905583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85e116-8fcf-4f3a-a344-ad5c40dada82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:51:59.923980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fd85e116-8fcf-4f3a-a344-ad5c40dada82 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:51:59.925154 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fd85e116-8fcf-4f3a-a344-ad5c40dada82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:51:59.989641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 428/862] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:51:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:52:00.261080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52afa7-1d05-4be5-ab7f-58c5eeee9217 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:00.316526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:00.358645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52afa7-1d05-4be5-ab7f-58c5eeee9217 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:00.390530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b51f4-2ac3-4972-ad96-71103e0cc275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:00.391133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b51f4-2ac3-4972-ad96-71103e0cc275 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:00.391437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b51f4-2ac3-4972-ad96-71103e0cc275 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:00.391708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b51f4-2ac3-4972-ad96-71103e0cc275 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:00.392789 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b51f4-2ac3-4972-ad96-71103e0cc275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:00.394192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b51f4-2ac3-4972-ad96-71103e0cc275 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:00.395754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 435/863] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:52:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:00.401430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85e116-8fcf-4f3a-a344-ad5c40dada82 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:00.413090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:00.497954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85e116-8fcf-4f3a-a344-ad5c40dada82 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:00.530972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb7ad5-9adf-4d03-9032-caf9f69a98b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:00.531592 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb7ad5-9adf-4d03-9032-caf9f69a98b9 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:00.531865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb7ad5-9adf-4d03-9032-caf9f69a98b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:00.532129 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb7ad5-9adf-4d03-9032-caf9f69a98b9 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:00.532760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb7ad5-9adf-4d03-9032-caf9f69a98b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:00.534170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb7ad5-9adf-4d03-9032-caf9f69a98b9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:00.554588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 429/864] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:52:00 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:52:02.661957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e217b-964b-4d19-a282-fe95511870ea None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:02.662481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e217b-964b-4d19-a282-fe95511870ea None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:02.662481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e217b-964b-4d19-a282-fe95511870ea None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:02.663836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 436/865] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:52:02.668779 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d45c85-0400-48b2-906d-2e11ff8c6bfc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:02.669145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d45c85-0400-48b2-906d-2e11ff8c6bfc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:02.669309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d45c85-0400-48b2-906d-2e11ff8c6bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:02.687027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-36d45c85-0400-48b2-906d-2e11ff8c6bfc None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:52:02.688214 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-36d45c85-0400-48b2-906d-2e11ff8c6bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:52:02.744293 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:02.752213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 430/866] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:52:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:52:03.018805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e217b-964b-4d19-a282-fe95511870ea None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:03.116196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e217b-964b-4d19-a282-fe95511870ea None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:03.138434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:03.148754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:03.149367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:03.149641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:03.149909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:03.159845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:52:03.165030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:52:03.166097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:52:03.167106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 437/867] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:52:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:52:03.172067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d45c85-0400-48b2-906d-2e11ff8c6bfc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:03.264687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d45c85-0400-48b2-906d-2e11ff8c6bfc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:03.296426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:03.296950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:03.297223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:03.297490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:03.298128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:52:03.298424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:03.299811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:03.307473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 431/868] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:52:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:03.314009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:03.408805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b2aa-1303-4bfc-8dca-03639dbfe432 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:03.440390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:03.440996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:03.441275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:03.442069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] PATH_INFO: `/v3/users/92461906bab3426084400eb57572fdad` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:03.454337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=92461906bab3426084400eb57572fdad)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:03.455774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:03.457891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 438/869] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:52:03 2022] GET /identity/v3/users/92461906bab3426084400eb57572fdad => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:03.463514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:03.518827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:03.560038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efe44f-ffa0-452a-947b-7078a583c652 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:03.592567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c27a4b-8031-4dbe-9f67-8613e20b1eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:03.593179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c27a4b-8031-4dbe-9f67-8613e20b1eaa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:03.593455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c27a4b-8031-4dbe-9f67-8613e20b1eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:03.593725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c27a4b-8031-4dbe-9f67-8613e20b1eaa None admin] PATH_INFO: `/v3/projects/9c0e1bce70414c2ab8b2b5f925332563` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:03.594807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c27a4b-8031-4dbe-9f67-8613e20b1eaa None admin] RBAC: Authorizing `identity:get_project(project_id=9c0e1bce70414c2ab8b2b5f925332563)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:03.596249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c27a4b-8031-4dbe-9f67-8613e20b1eaa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:03.597881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 432/870] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:52:03 2022] GET /identity/v3/projects/9c0e1bce70414c2ab8b2b5f925332563 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:03.603834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:03.645317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:03.698955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969f7e9-445f-41ec-acac-fae991fd3cc0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:03.730077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ec944-0016-4508-acf3-13847c07fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:03.730706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ec944-0016-4508-acf3-13847c07fa18 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:03.730977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ec944-0016-4508-acf3-13847c07fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:03.731271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ec944-0016-4508-acf3-13847c07fa18 None admin] PATH_INFO: `/v3/projects/9c0e1bce70414c2ab8b2b5f925332563/users/92461906bab3426084400eb57572fdad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:03.733099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ec944-0016-4508-acf3-13847c07fa18 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c0e1bce70414c2ab8b2b5f925332563, user_id=92461906bab3426084400eb57572fdad, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:03.734499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ec944-0016-4508-acf3-13847c07fa18 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:03.747059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 439/871] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:52:03 2022] PUT /identity/v3/projects/9c0e1bce70414c2ab8b2b5f925332563/users/92461906bab3426084400eb57572fdad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:52:05.813117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80457bbb-62bf-486d-935d-36c37c9d69b4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:05.813647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80457bbb-62bf-486d-935d-36c37c9d69b4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:05.813647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80457bbb-62bf-486d-935d-36c37c9d69b4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:05.815241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 433/872] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:52:05.820837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f2835-2d7e-4417-b766-d30dc8cef2e4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:05.821248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f2835-2d7e-4417-b766-d30dc8cef2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:05.821334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f2835-2d7e-4417-b766-d30dc8cef2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:05.839552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-673f2835-2d7e-4417-b766-d30dc8cef2e4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:52:05.840717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-673f2835-2d7e-4417-b766-d30dc8cef2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:52:05.905926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 440/873] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:52:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:52:06.171831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80457bbb-62bf-486d-935d-36c37c9d69b4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:06.270014 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80457bbb-62bf-486d-935d-36c37c9d69b4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:06.294535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:06.303929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:06.304536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:06.304813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:06.305109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:06.315076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:52:06.319810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:52:06.320890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:52:06.321935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 434/874] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:52:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:52:06.326827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f2835-2d7e-4417-b766-d30dc8cef2e4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:06.356296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:06.423630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f2835-2d7e-4417-b766-d30dc8cef2e4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:06.456324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:06.456920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:06.457194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:06.457465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:06.458118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:52:06.458413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:06.459797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:06.467609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 441/875] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:52:06.473916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:06.569092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34099103-a2d5-484e-9beb-94f702611ac6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:06.601649 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:06.602260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:06.602529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:06.602792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] PATH_INFO: `/v3/users/92461906bab3426084400eb57572fdad` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:06.603896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] RBAC: Authorizing `identity:get_user(user_id=92461906bab3426084400eb57572fdad)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:06.605343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:06.606891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 435/876] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:52:06 2022] GET /identity/v3/users/92461906bab3426084400eb57572fdad => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:06.612600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:06.701717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:06.708852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8b264-04ec-4fe2-8636-61c469486745 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:06.740774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:06.741359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:06.741633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:06.741901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:06.805029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:06.806458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:06.892379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:06.900423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 442/877] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:52:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 17 06:52:06.906249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:06.959696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:07.002917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f3cd0-0d7f-47ed-acc3-232c304d80ed None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:07.035807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05b7d7-d427-4d0f-948c-93a506037b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:07.036410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05b7d7-d427-4d0f-948c-93a506037b58 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:07.036685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05b7d7-d427-4d0f-948c-93a506037b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:07.036952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05b7d7-d427-4d0f-948c-93a506037b58 None admin] PATH_INFO: `/v3/projects/9c0e1bce70414c2ab8b2b5f925332563` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:07.038024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05b7d7-d427-4d0f-948c-93a506037b58 None admin] RBAC: Authorizing `identity:get_project(project_id=9c0e1bce70414c2ab8b2b5f925332563)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:07.039489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05b7d7-d427-4d0f-948c-93a506037b58 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:07.041071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 436/878] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:52:06 2022] GET /identity/v3/projects/9c0e1bce70414c2ab8b2b5f925332563 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:07.047076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:07.141419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edad40a-fa92-4eaf-b9b2-922c4939ab30 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:07.174226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececfc22-fcb4-41e1-998c-a98564a766ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:07.174812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececfc22-fcb4-41e1-998c-a98564a766ea None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:07.175090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececfc22-fcb4-41e1-998c-a98564a766ea None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:07.175387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececfc22-fcb4-41e1-998c-a98564a766ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:07.176125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececfc22-fcb4-41e1-998c-a98564a766ea None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=9c0e1bce70414c2ab8b2b5f925332563, user.id=92461906bab3426084400eb57572fdad) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:52:07.176429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececfc22-fcb4-41e1-998c-a98564a766ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:07.177842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececfc22-fcb4-41e1-998c-a98564a766ea None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:07.183366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 443/879] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:52:07 2022] GET /identity/v3/role_assignments?user.id=92461906bab3426084400eb57572fdad&scope.project.id=9c0e1bce70414c2ab8b2b5f925332563&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:09.293982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a3ad-07cd-4cfb-9764-9ac90311e421 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:09.294492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a3ad-07cd-4cfb-9764-9ac90311e421 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:09.294593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a3ad-07cd-4cfb-9764-9ac90311e421 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:09.295884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 437/880] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:52:09.301117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd44c98-91fc-4399-8460-8c56f3898cb7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:09.301529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd44c98-91fc-4399-8460-8c56f3898cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:09.301642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd44c98-91fc-4399-8460-8c56f3898cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:09.319839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0cd44c98-91fc-4399-8460-8c56f3898cb7 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:52:09.320989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0cd44c98-91fc-4399-8460-8c56f3898cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:52:09.370958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:09.384870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 444/881] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:52:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:52:09.649291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a3ad-07cd-4cfb-9764-9ac90311e421 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:09.745606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a3ad-07cd-4cfb-9764-9ac90311e421 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:09.767104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:09.779082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40299f-afce-4036-84aa-911d2f305a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:09.779698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40299f-afce-4036-84aa-911d2f305a4d None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:09.779972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40299f-afce-4036-84aa-911d2f305a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:09.780240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40299f-afce-4036-84aa-911d2f305a4d None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:09.780882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40299f-afce-4036-84aa-911d2f305a4d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:09.782232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40299f-afce-4036-84aa-911d2f305a4d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:09.797149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 438/882] 10.222.0.36 () {60 vars in 1138 bytes} [Mon Oct 17 06:52:09 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:52:11.926946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080d90d-a950-47b6-b4c4-fa7a01b2fe7d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:11.927498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080d90d-a950-47b6-b4c4-fa7a01b2fe7d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:11.927498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080d90d-a950-47b6-b4c4-fa7a01b2fe7d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:11.928851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 445/883] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:52:11.934234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb404a-54c8-4d0e-bd5c-0ab5cedd6638 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:11.934640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb404a-54c8-4d0e-bd5c-0ab5cedd6638 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:11.934772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb404a-54c8-4d0e-bd5c-0ab5cedd6638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:11.952532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bbdb404a-54c8-4d0e-bd5c-0ab5cedd6638 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:52:11.953695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bbdb404a-54c8-4d0e-bd5c-0ab5cedd6638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:52:12.016738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 439/884] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:52:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:52:12.285857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080d90d-a950-47b6-b4c4-fa7a01b2fe7d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:12.387006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080d90d-a950-47b6-b4c4-fa7a01b2fe7d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:12.422206 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62ae64-de12-4b5b-a9d6-b29837b311da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:12.422814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ae64-de12-4b5b-a9d6-b29837b311da None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:12.423092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ae64-de12-4b5b-a9d6-b29837b311da None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:12.423392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ae64-de12-4b5b-a9d6-b29837b311da None admin] PATH_INFO: `/v3/domains/df540ca61f33431cab129ef946128b7c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:12.433953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62ae64-de12-4b5b-a9d6-b29837b311da None admin] RBAC: Authorizing `identity:get_domain(domain_id=df540ca61f33431cab129ef946128b7c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:12.435649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62ae64-de12-4b5b-a9d6-b29837b311da None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:12.437206 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 446/885] 10.222.0.36 () {56 vars in 1183 bytes} [Mon Oct 17 06:52:12 2022] GET /identity/v3/domains/df540ca61f33431cab129ef946128b7c => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:12.442941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb404a-54c8-4d0e-bd5c-0ab5cedd6638 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:12.536895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb404a-54c8-4d0e-bd5c-0ab5cedd6638 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:12.549132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:12.571304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779693f-edd2-489f-8ebc-b297b1768481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:12.571904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779693f-edd2-489f-8ebc-b297b1768481 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:12.572185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779693f-edd2-489f-8ebc-b297b1768481 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:12.572457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779693f-edd2-489f-8ebc-b297b1768481 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:12.573094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779693f-edd2-489f-8ebc-b297b1768481 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:12.574520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779693f-edd2-489f-8ebc-b297b1768481 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:12.602161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 440/886] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:52:12 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:52:14.755276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523f3f-2fe0-419b-87da-89e237399bde None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:14.755854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523f3f-2fe0-419b-87da-89e237399bde None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:14.755854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523f3f-2fe0-419b-87da-89e237399bde None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:14.757148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 447/887] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:52:14.762380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27d44-8f54-4025-a6ff-bba759aa6b28 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:14.762818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27d44-8f54-4025-a6ff-bba759aa6b28 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:14.762921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27d44-8f54-4025-a6ff-bba759aa6b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:14.781773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-36f27d44-8f54-4025-a6ff-bba759aa6b28 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:52:14.782940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-36f27d44-8f54-4025-a6ff-bba759aa6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:52:14.846422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 441/888] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:52:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:52:15.114444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b523f3f-2fe0-419b-87da-89e237399bde None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:15.162851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:15.212534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b523f3f-2fe0-419b-87da-89e237399bde None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:15.244166 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf28325-a770-4e29-9317-6bd664b5787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:15.244767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28325-a770-4e29-9317-6bd664b5787f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:15.245039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28325-a770-4e29-9317-6bd664b5787f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:15.245305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28325-a770-4e29-9317-6bd664b5787f None admin] PATH_INFO: `/v3/domains/df540ca61f33431cab129ef946128b7c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:15.246378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf28325-a770-4e29-9317-6bd664b5787f None admin] RBAC: Authorizing `identity:get_domain(domain_id=df540ca61f33431cab129ef946128b7c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:15.248045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf28325-a770-4e29-9317-6bd664b5787f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:15.249562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 448/889] 10.222.0.36 () {56 vars in 1183 bytes} [Mon Oct 17 06:52:15 2022] GET /identity/v3/domains/df540ca61f33431cab129ef946128b7c => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:15.255239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27d44-8f54-4025-a6ff-bba759aa6b28 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:15.342713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:15.350061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27d44-8f54-4025-a6ff-bba759aa6b28 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:15.382784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a05f5-90b4-47c4-acc5-6e088af2f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:15.383344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a05f5-90b4-47c4-acc5-6e088af2f9b5 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:15.383621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a05f5-90b4-47c4-acc5-6e088af2f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:15.383887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a05f5-90b4-47c4-acc5-6e088af2f9b5 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:15.384525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a05f5-90b4-47c4-acc5-6e088af2f9b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:15.385946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a05f5-90b4-47c4-acc5-6e088af2f9b5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:15.405451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 442/890] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:52:15 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:52:17.528960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df232885-bb6b-41d6-83fa-5be31587cd69 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:17.529465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df232885-bb6b-41d6-83fa-5be31587cd69 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:17.529465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df232885-bb6b-41d6-83fa-5be31587cd69 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:17.530740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 449/891] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:52:17.535847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40de6d-852e-44fe-b347-c0719018bd60 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:17.536222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40de6d-852e-44fe-b347-c0719018bd60 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:17.536389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40de6d-852e-44fe-b347-c0719018bd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:17.553933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7d40de6d-852e-44fe-b347-c0719018bd60 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:52:17.555100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7d40de6d-852e-44fe-b347-c0719018bd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:52:17.619489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 443/892] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:52:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:52:17.883120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df232885-bb6b-41d6-83fa-5be31587cd69 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:17.979860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df232885-bb6b-41d6-83fa-5be31587cd69 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:17.995698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:18.013108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:18.013709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:18.013978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:18.014241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:18.023781 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:52:18.028406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:52:18.029572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:52:18.030690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 450/893] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:52:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:52:18.035337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40de6d-852e-44fe-b347-c0719018bd60 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:18.111219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:18.131706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40de6d-852e-44fe-b347-c0719018bd60 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:18.164733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:18.165362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:18.165640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:18.165912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:18.166562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:52:18.166865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:18.168268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:18.176681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 444/894] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:52:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:18.182375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:18.276430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada90a4-2140-4986-b259-d92fa48dcb14 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:18.308527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:18.309123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:18.309394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:18.309660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] PATH_INFO: `/v3/users/b62a88a0f8534be58625a079c5c75f1d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:18.322126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] RBAC: Authorizing `identity:get_user(user_id=b62a88a0f8534be58625a079c5c75f1d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:18.323562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:18.325090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 451/895] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:52:18 2022] GET /identity/v3/users/b62a88a0f8534be58625a079c5c75f1d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:18.330710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:18.426902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2eea1e-843d-447d-bb90-5275c3dbd7e8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:18.459487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f76a41-90bc-4df6-8a39-28ace1156c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:18.460103 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76a41-90bc-4df6-8a39-28ace1156c7d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:18.460379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76a41-90bc-4df6-8a39-28ace1156c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:18.460651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76a41-90bc-4df6-8a39-28ace1156c7d None admin] PATH_INFO: `/v3/projects/69f85c64f8e44586a89d8cf14c96827f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:18.461741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f76a41-90bc-4df6-8a39-28ace1156c7d None admin] RBAC: Authorizing `identity:get_project(project_id=69f85c64f8e44586a89d8cf14c96827f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:18.463166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f76a41-90bc-4df6-8a39-28ace1156c7d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:18.464796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 445/896] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:52:18 2022] GET /identity/v3/projects/69f85c64f8e44586a89d8cf14c96827f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:18.470611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:18.503747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:18.566321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee807fb6-8812-46cf-900e-aaf4984b7ffe None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:18.598130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378b029-4ae4-4d8a-96ed-42688139f3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:18.598761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378b029-4ae4-4d8a-96ed-42688139f3e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:18.599034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378b029-4ae4-4d8a-96ed-42688139f3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:18.599329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378b029-4ae4-4d8a-96ed-42688139f3e4 None admin] PATH_INFO: `/v3/projects/69f85c64f8e44586a89d8cf14c96827f/users/b62a88a0f8534be58625a079c5c75f1d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:18.601150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378b029-4ae4-4d8a-96ed-42688139f3e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=69f85c64f8e44586a89d8cf14c96827f, user_id=b62a88a0f8534be58625a079c5c75f1d, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:18.602544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378b029-4ae4-4d8a-96ed-42688139f3e4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:18.616013 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 452/897] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:52:18 2022] PUT /identity/v3/projects/69f85c64f8e44586a89d8cf14c96827f/users/b62a88a0f8534be58625a079c5c75f1d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:52:20.702516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004376f-62d7-4089-a72c-addf8293f2d7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:20.703055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004376f-62d7-4089-a72c-addf8293f2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:20.703055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004376f-62d7-4089-a72c-addf8293f2d7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:20.704409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 446/898] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:52:20.710207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053c45-3cbe-4073-9954-6d8892b60d17 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:20.710581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053c45-3cbe-4073-9954-6d8892b60d17 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:20.710738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053c45-3cbe-4073-9954-6d8892b60d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:20.728141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-be053c45-3cbe-4073-9954-6d8892b60d17 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:52:20.729291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-be053c45-3cbe-4073-9954-6d8892b60d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:52:20.791264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 453/899] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:52:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:52:21.056440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004376f-62d7-4089-a72c-addf8293f2d7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:21.069144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:21.154435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004376f-62d7-4089-a72c-addf8293f2d7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:21.188073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:21.188678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:21.188949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:21.189212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:21.199511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:52:21.204099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:52:21.205227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:52:21.206831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 447/900] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:52:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:52:21.211664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be053c45-3cbe-4073-9954-6d8892b60d17 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:21.234794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:21.305598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be053c45-3cbe-4073-9954-6d8892b60d17 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:21.337974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:21.338501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:21.338773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:21.339038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:21.339710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:52:21.340005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:21.341355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:21.348875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 454/901] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:52:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:21.354583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:21.419911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:21.449682 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f73de-d793-42be-9f4c-33010178e887 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:21.481391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:21.481988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:21.482256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:21.482520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] PATH_INFO: `/v3/users/b62a88a0f8534be58625a079c5c75f1d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:21.483616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] RBAC: Authorizing `identity:get_user(user_id=b62a88a0f8534be58625a079c5c75f1d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:21.485022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:21.486538 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 448/902] 10.222.0.36 () {56 vars in 1177 bytes} [Mon Oct 17 06:52:21 2022] GET /identity/v3/users/b62a88a0f8534be58625a079c5c75f1d => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:21.492303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:21.573969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:21.588358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1d604-d015-47b9-88dd-f178ba593190 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:21.620021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:21.620605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:21.620876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:21.621140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:21.684347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:21.686000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:21.764290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:21.779094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 455/903] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:52:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:52:21.784970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:21.879352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb220cb8-5c9c-486c-beaa-78451fe784ed None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:21.911257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4874f4-91bb-4a7d-aadc-63d044de9ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:21.911861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4874f4-91bb-4a7d-aadc-63d044de9ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:21.912132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4874f4-91bb-4a7d-aadc-63d044de9ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:21.912399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4874f4-91bb-4a7d-aadc-63d044de9ed9 None admin] PATH_INFO: `/v3/projects/69f85c64f8e44586a89d8cf14c96827f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:21.913473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4874f4-91bb-4a7d-aadc-63d044de9ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=69f85c64f8e44586a89d8cf14c96827f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:21.914885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4874f4-91bb-4a7d-aadc-63d044de9ed9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:21.916480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 449/904] 10.222.0.36 () {56 vars in 1186 bytes} [Mon Oct 17 06:52:21 2022] GET /identity/v3/projects/69f85c64f8e44586a89d8cf14c96827f => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:21.922519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:52:22.016053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358de70-e03e-4dc5-9dfc-56bc41ee5f22 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:52:22.048150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3631a96-80de-418b-ade6-b2edea5e0cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:52:22.048729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3631a96-80de-418b-ade6-b2edea5e0cae None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:52:22.049005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3631a96-80de-418b-ade6-b2edea5e0cae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:52:22.049278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3631a96-80de-418b-ade6-b2edea5e0cae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:52:22.049966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3631a96-80de-418b-ade6-b2edea5e0cae None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=69f85c64f8e44586a89d8cf14c96827f, user.id=b62a88a0f8534be58625a079c5c75f1d) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:52:22.050266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3631a96-80de-418b-ade6-b2edea5e0cae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:52:22.051694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3631a96-80de-418b-ade6-b2edea5e0cae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:52:22.057123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 456/905] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:52:21 2022] GET /identity/v3/role_assignments?user.id=b62a88a0f8534be58625a079c5c75f1d&scope.project.id=69f85c64f8e44586a89d8cf14c96827f&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:52:31.906732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:52:32.054685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=75 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:16.733415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8b797-1639-4a69-93a9-cfc8da0345ed None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:16.734604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8b797-1639-4a69-93a9-cfc8da0345ed None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:16.735524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8b797-1639-4a69-93a9-cfc8da0345ed None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:16.737134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 450/906] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:16.743141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44266706-0560-4854-81c3-6ca9cee22d3e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:16.743666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44266706-0560-4854-81c3-6ca9cee22d3e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:16.743752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44266706-0560-4854-81c3-6ca9cee22d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:16.765393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-44266706-0560-4854-81c3-6ca9cee22d3e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:16.766719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-44266706-0560-4854-81c3-6ca9cee22d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:16.871030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 457/907] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:53:17.153462 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8b797-1639-4a69-93a9-cfc8da0345ed None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:17.271581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8b797-1639-4a69-93a9-cfc8da0345ed None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:17.310877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:17.311569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:17.311844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:17.312716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:17.325773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:17.327234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:17.344502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:17.346019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:17.347098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:17.347098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:17.347098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:17.350891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 451/908] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:17.352395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44266706-0560-4854-81c3-6ca9cee22d3e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:17.475645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44266706-0560-4854-81c3-6ca9cee22d3e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:17.509631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:17.518974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c19bf2-ee76-4340-a7ac-4a8bd08bd88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:17.519657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19bf2-ee76-4340-a7ac-4a8bd08bd88f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:17.520007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19bf2-ee76-4340-a7ac-4a8bd08bd88f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:17.520290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19bf2-ee76-4340-a7ac-4a8bd08bd88f None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:17.521076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19bf2-ee76-4340-a7ac-4a8bd08bd88f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:17.521371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19bf2-ee76-4340-a7ac-4a8bd08bd88f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:17.522907 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19bf2-ee76-4340-a7ac-4a8bd08bd88f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:17.538027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 458/909] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:17.543503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:17.605063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:17.658302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32841983-7c2d-4e84-be47-0fe77b9559d4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:17.703615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a3cda-7534-482f-940b-816c93853833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:17.704399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3cda-7534-482f-940b-816c93853833 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:17.704737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3cda-7534-482f-940b-816c93853833 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:17.705111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3cda-7534-482f-940b-816c93853833 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:17.705821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a3cda-7534-482f-940b-816c93853833 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:17.707453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a3cda-7534-482f-940b-816c93853833 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:17.754623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 452/910] 10.222.0.36 () {60 vars in 1132 bytes} [Mon Oct 17 06:53:17 2022] POST /identity/v3/users => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:53:19.973220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d09e-e99e-42a6-804c-89e8ee9d6941 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:19.973765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d09e-e99e-42a6-804c-89e8ee9d6941 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:19.973765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d09e-e99e-42a6-804c-89e8ee9d6941 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:19.975253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 459/911] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:19.980617 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a0c1a-a149-4918-beeb-a72f60191b63 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:19.981650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a0c1a-a149-4918-beeb-a72f60191b63 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:19.981650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a0c1a-a149-4918-beeb-a72f60191b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:20.001713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a70a0c1a-a149-4918-beeb-a72f60191b63 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:20.002880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a70a0c1a-a149-4918-beeb-a72f60191b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:20.077429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 453/912] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:20.346283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d09e-e99e-42a6-804c-89e8ee9d6941 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:20.449488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d09e-e99e-42a6-804c-89e8ee9d6941 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:20.492610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:20.493932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:20.494217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:20.494620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:20.508575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:53:20.513921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:20.516716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:20.517795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 460/913] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:53:20.522095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a0c1a-a149-4918-beeb-a72f60191b63 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:20.569279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:20.646362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a0c1a-a149-4918-beeb-a72f60191b63 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:20.688808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:20.689939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:20.689939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:20.689939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:20.690537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:20.690839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:20.692414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:20.701746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 454/914] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:53:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:53:20.707811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:20.766371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:20.825800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef6022-9275-40c9-b8b8-f84cc9e68696 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:20.876130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:20.876917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:20.877377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:20.877799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:20.889189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:20.891437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:20.908182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:20.909456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:20.910353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:20.910353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:20.910353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:20.910353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 461/915] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:20.915260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:21.037680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:21.042631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a61e6-d8ff-4749-8908-b7f196bf1798 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:21.080554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e288668-1c17-445e-b757-68289461212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:21.081161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e288668-1c17-445e-b757-68289461212c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:21.081435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e288668-1c17-445e-b757-68289461212c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:21.081980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e288668-1c17-445e-b757-68289461212c None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:21.082704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e288668-1c17-445e-b757-68289461212c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:21.083008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e288668-1c17-445e-b757-68289461212c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:21.084477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e288668-1c17-445e-b757-68289461212c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:21.097616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 455/916] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:21.103914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:21.218747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610ba68-9c5b-4b55-a44e-8fa4c4985673 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:21.220255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:21.256961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:21.257587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:21.257856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:21.258126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:21.266482 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:21.267941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:53:21.279169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:21.280300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:21.281320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 462/917] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:53:21.286349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e288668-1c17-445e-b757-68289461212c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:21.398309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e288668-1c17-445e-b757-68289461212c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:21.435975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:21.437496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:21.438017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:21.438461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:21.439571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:21.440155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:21.441992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:21.461252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 456/918] 10.222.0.36 () {56 vars in 1143 bytes} [Mon Oct 17 06:53:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:53:21.467225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:21.581285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77087d3-6a10-486e-8615-e61a0e2649ef None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:21.622990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:21.623794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:21.624186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:21.624579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:21.635390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:21.636933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:21.655147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:21.656924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:21.658052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:21.658052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:21.658052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:21.659342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 463/919] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:53:21.664809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:21.685120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:21.796977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaea56-e389-4573-987c-4e8c9ac6757b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:21.834030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:21.834627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:21.834964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:21.835373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:21.836066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:21.836487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:21.838027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:21.860856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:21.864066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 457/920] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:21.878860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:21.980699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d038096-2a51-4744-b1cd-d22f40d1a7d0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:22.020941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:22.021652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:22.021998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:22.022337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:22.029033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:22.031059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:53:22.044067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:22.045187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:22.047106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:22.051023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 464/921] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:22.052148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:22.139741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:22.176059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fc0f4-5f78-4ee9-ba72-dc7b371bcedd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:22.216198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab1ceb-6060-4a67-83c7-f5b673066eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:22.216844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab1ceb-6060-4a67-83c7-f5b673066eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:22.217112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab1ceb-6060-4a67-83c7-f5b673066eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:22.217448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab1ceb-6060-4a67-83c7-f5b673066eb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:22.218662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab1ceb-6060-4a67-83c7-f5b673066eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:22.219051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab1ceb-6060-4a67-83c7-f5b673066eb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:22.220587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab1ceb-6060-4a67-83c7-f5b673066eb4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:22.236858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 458/922] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:22.243646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:22.334436 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:22.361981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07f4f-1ecc-4f57-94ca-05835b8c137a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:22.398410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de649a4-bde4-4b07-a103-bfc864fb1d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:22.399072 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de649a4-bde4-4b07-a103-bfc864fb1d44 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:22.399400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de649a4-bde4-4b07-a103-bfc864fb1d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:22.399678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de649a4-bde4-4b07-a103-bfc864fb1d44 None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/522ff98d9ecb4d90bd97a19f71a400c8/roles/f2f75648d0384a4cac1041775239be27` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:22.415883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de649a4-bde4-4b07-a103-bfc864fb1d44 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=522ff98d9ecb4d90bd97a19f71a400c8, role_id=f2f75648d0384a4cac1041775239be27)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:22.417406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de649a4-bde4-4b07-a103-bfc864fb1d44 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:22.432398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 465/923] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:53:22 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/522ff98d9ecb4d90bd97a19f71a400c8/roles/f2f75648d0384a4cac1041775239be27 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:53:24.711638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae118db-8ffd-4015-b478-587a789d84d4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:24.712177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae118db-8ffd-4015-b478-587a789d84d4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:24.712177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae118db-8ffd-4015-b478-587a789d84d4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:24.713534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 459/924] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:53:24.719872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071e18f-d1f1-4cd9-af0a-9a6552c53d65 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:24.720288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071e18f-d1f1-4cd9-af0a-9a6552c53d65 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:24.720408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071e18f-d1f1-4cd9-af0a-9a6552c53d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:24.737964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3071e18f-d1f1-4cd9-af0a-9a6552c53d65 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:24.739125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3071e18f-d1f1-4cd9-af0a-9a6552c53d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:24.812863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 466/925] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:25.095306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae118db-8ffd-4015-b478-587a789d84d4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:25.207401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae118db-8ffd-4015-b478-587a789d84d4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:25.241342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:25.242005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:25.242281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:25.242588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:25.250468 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:25.255318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:53:25.261110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:25.262093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:25.262956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 460/926] 10.222.0.36 () {56 vars in 1102 bytes} [Mon Oct 17 06:53:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:25.268614 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071e18f-d1f1-4cd9-af0a-9a6552c53d65 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:25.330739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:25.378956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071e18f-d1f1-4cd9-af0a-9a6552c53d65 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:25.417747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:25.418434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:25.418705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:25.418991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:25.419740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:25.420068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:25.422497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:25.430644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 467/927] 10.222.0.36 () {56 vars in 1103 bytes} [Mon Oct 17 06:53:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:25.437587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:25.544864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa612-def7-423a-97f2-71e5d8a398da None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:25.584783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:25.585617 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:25.586020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:25.586386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:25.597833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:25.599338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:25.615231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:25.616478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:25.617500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:25.617500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:25.617500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:25.617500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 461/928] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:25.622606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:25.729811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda949f-0c5e-42d1-98a8-afa80d38ce86 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:25.748341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:25.766204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:25.766804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:25.767077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:25.767409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:25.768087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:25.768391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:25.769936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:25.782894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 468/929] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:25.788893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:25.843862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:25.906061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e1f8-3c94-4c3e-b805-1587c2475001 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:25.940496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:25.941112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:25.941384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:25.941648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:25.948962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:25.950384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:53:25.962073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:25.963335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:25.964273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 462/930] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:25.969817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:26.076597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea0b8c-0df6-4f36-80a7-2c5058adc4b0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:26.115498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:26.116101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:26.116465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:26.116809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:26.117620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:26.117961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:26.119817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:26.138329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 469/931] 10.222.0.36 () {56 vars in 1143 bytes} [Mon Oct 17 06:53:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:26.145102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:26.259195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2097d6-7e63-4163-92cd-596a9c7238a9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:26.271491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:26.301278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:26.301937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:26.302347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:26.302625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:26.377130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:26.378548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:26.491630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 463/932] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:53:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:53:26.497610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:26.533334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:26.606194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a9748-d110-4784-9472-10e4c8beb402 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:26.643971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:26.644617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:26.645001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:26.645413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:26.655420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:26.656948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:26.672331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:26.673598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:26.674712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:26.674712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:26.674712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:26.674712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 470/933] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:26.680556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:26.694956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:26.783883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071939dd-8a43-4152-96c0-c3a2def4641c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:26.820522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:26.821244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:26.821427 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:26.821783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:26.822554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:26.822854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:26.824362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:26.838572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 464/934] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:26.847919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:26.951050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:26.966707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d52e1-3156-4986-92d6-acb78d2f8421 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:27.001600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:27.002215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:27.002495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:27.002762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:27.009672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:27.011394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:53:27.024065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:27.025165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:27.026261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:27.031502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 471/935] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:27.033074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:27.112231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:27.144629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e12d-5670-4448-8ff1-5ce4de949a17 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:27.185954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e2d6-97be-49ae-b37c-70e92ccbdd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:27.187126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e2d6-97be-49ae-b37c-70e92ccbdd6e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:27.187428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e2d6-97be-49ae-b37c-70e92ccbdd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:27.187699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e2d6-97be-49ae-b37c-70e92ccbdd6e None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:27.188378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e2d6-97be-49ae-b37c-70e92ccbdd6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:27.188678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e2d6-97be-49ae-b37c-70e92ccbdd6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:27.191320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e2d6-97be-49ae-b37c-70e92ccbdd6e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:27.204523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 465/936] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:27.212258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:27.320471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b042c-b9ef-4267-bbb1-2013df27016b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:27.358153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf5738-5789-4537-b880-a68587fc8899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:27.359325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5738-5789-4537-b880-a68587fc8899 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:27.359604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5738-5789-4537-b880-a68587fc8899 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:27.359871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5738-5789-4537-b880-a68587fc8899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:27.360581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf5738-5789-4537-b880-a68587fc8899 None admin] RBAC: Adding query filter params (role.id=f2f75648d0384a4cac1041775239be27, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=522ff98d9ecb4d90bd97a19f71a400c8) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:27.360885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf5738-5789-4537-b880-a68587fc8899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:27.362313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf5738-5789-4537-b880-a68587fc8899 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:27.368087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 472/937] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:53:27 2022] GET /identity/v3/role_assignments?user.id=522ff98d9ecb4d90bd97a19f71a400c8&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=f2f75648d0384a4cac1041775239be27 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:29.599495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0cf9-8721-4193-b5b7-0387c8b5eec9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:29.600043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0cf9-8721-4193-b5b7-0387c8b5eec9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:29.600043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0cf9-8721-4193-b5b7-0387c8b5eec9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:29.601277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 466/938] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:29.606819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b3e93-b462-43cd-8af0-901606d22163 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:29.607331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b3e93-b462-43cd-8af0-901606d22163 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:29.607473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b3e93-b462-43cd-8af0-901606d22163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:29.626883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7d6b3e93-b462-43cd-8af0-901606d22163 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:29.628090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7d6b3e93-b462-43cd-8af0-901606d22163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:29.631263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:29.698282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 473/939] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:29.982344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd0cf9-8721-4193-b5b7-0387c8b5eec9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:30.094859 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd0cf9-8721-4193-b5b7-0387c8b5eec9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:30.127516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:30.133790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:30.134437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:30.134691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:30.134957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:30.149311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:53:30.154111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:30.155346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:30.156727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 467/940] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:30.161517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b3e93-b462-43cd-8af0-901606d22163 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:30.271292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b3e93-b462-43cd-8af0-901606d22163 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:30.307747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:30.308348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:30.308902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:30.309173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:30.310003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:30.310477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:30.313054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:30.321421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 474/941] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:53:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:30.327282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:30.440352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848df20-8a8b-4dc1-bae7-37740e5165b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:30.473972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:30.474586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:30.474858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:30.475122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:30.486137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:30.487610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:30.502824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:30.503878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:30.504791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:30.504791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:30.504791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:30.504791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 468/942] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:30.510111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:30.513761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:30.624162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c205959-e779-4723-b9ff-0508e4dc1c6d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:30.658466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:30.659093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:30.659445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:30.659687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:30.660440 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:30.660643 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:30.662157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:30.675558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 475/943] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:30.681629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:30.735137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:30.796516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95205ec2-1b52-4fcc-8d43-cf466cba67c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:30.832338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:30.833065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:30.833341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:30.833613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:30.842679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:30.844394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:53:30.856472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:30.857758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:30.858743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 469/944] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:30.864261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:30.940118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:30.981464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693ae4-efad-4a89-9ce6-c29dc2908f4b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:31.018271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:31.019135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:31.019541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:31.019926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:31.020998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:31.021298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:31.022976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:31.043228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 476/945] 10.222.0.36 () {56 vars in 1143 bytes} [Mon Oct 17 06:53:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:31.049773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:31.160867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91d23-87b3-4d36-a646-c19f41a0f837 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:31.162410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:31.199844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:31.200397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:31.200676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:31.200948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:31.212212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:31.214106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:31.231761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:31.232771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:31.234400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:31.234400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:31.234400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:31.234400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 470/946] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:31.239746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:31.347395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4154b7-81cc-4000-b5da-273020b70579 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:31.370894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:31.387330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:31.387927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:31.388249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:31.388520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:31.389157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:31.389545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:31.391018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:31.403163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 477/947] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:31.410007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:31.524744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749e55c-3fa9-4ed5-b9f9-70fbc9da2546 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:31.560878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:31.561634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:31.561917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:31.562228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:31.569517 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:31.571174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:53:31.583225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:31.584323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:31.585425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 471/948] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:53:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:31.590571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:31.703079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101cfb9-320a-42f2-8cb3-44bf586b9733 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:31.740851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82445-1e09-478a-a61b-0ec5c5ed0c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:31.741454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82445-1e09-478a-a61b-0ec5c5ed0c7f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:31.741731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82445-1e09-478a-a61b-0ec5c5ed0c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:31.742001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82445-1e09-478a-a61b-0ec5c5ed0c7f None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:31.742685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82445-1e09-478a-a61b-0ec5c5ed0c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:31.742988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82445-1e09-478a-a61b-0ec5c5ed0c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:31.744446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82445-1e09-478a-a61b-0ec5c5ed0c7f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:31.758222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 478/949] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:31.766271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:31.785585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:31.897935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a56a44-978e-4562-8cb5-01194a5a3509 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:31.943339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263afd7-3767-406e-b36d-b100ee02a508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:31.944002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263afd7-3767-406e-b36d-b100ee02a508 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:31.944294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263afd7-3767-406e-b36d-b100ee02a508 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:31.944550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263afd7-3767-406e-b36d-b100ee02a508 None admin] PATH_INFO: `/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/522ff98d9ecb4d90bd97a19f71a400c8/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:31.946403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263afd7-3767-406e-b36d-b100ee02a508 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1f20ff4c5f24755b1f918e07e8b23fc, user_id=522ff98d9ecb4d90bd97a19f71a400c8, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:31.948130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263afd7-3767-406e-b36d-b100ee02a508 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:31.962224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 472/950] 10.222.0.36 () {58 vars in 1439 bytes} [Mon Oct 17 06:53:31 2022] PUT /identity/v3/projects/c1f20ff4c5f24755b1f918e07e8b23fc/users/522ff98d9ecb4d90bd97a19f71a400c8/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:53:34.492304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879523-8e64-40b7-a668-834e3451621d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:34.492858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879523-8e64-40b7-a668-834e3451621d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:34.492858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879523-8e64-40b7-a668-834e3451621d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:34.494076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 479/951] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:34.501501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73446e13-bc58-420d-9bc0-61fdd73b217a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:34.501501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73446e13-bc58-420d-9bc0-61fdd73b217a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:34.501501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73446e13-bc58-420d-9bc0-61fdd73b217a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:34.522935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-73446e13-bc58-420d-9bc0-61fdd73b217a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:34.524345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-73446e13-bc58-420d-9bc0-61fdd73b217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:34.597702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 473/952] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:34.917517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a879523-8e64-40b7-a668-834e3451621d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:34.966492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:35.036562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a879523-8e64-40b7-a668-834e3451621d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:35.076375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:35.077576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:35.077883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:35.078245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:35.089857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:53:35.096210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:35.097391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:35.098600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 480/953] 10.222.0.36 () {56 vars in 1096 bytes} [Mon Oct 17 06:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:53:35.104698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73446e13-bc58-420d-9bc0-61fdd73b217a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:35.107889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:35.224066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73446e13-bc58-420d-9bc0-61fdd73b217a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:35.265618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:35.266305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:35.266643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:35.266985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:35.267778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:35.268143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:35.269631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:35.279149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 474/954] 10.222.0.36 () {56 vars in 1099 bytes} [Mon Oct 17 06:53:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:35.285797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:35.388007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:35.391455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339cf58-a1ab-4e6b-8dac-d8cc97fec842 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:35.428551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:35.429162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:35.429436 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:35.429704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:35.440888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:35.442310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:35.458068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:35.459250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:35.460657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:35.460657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:35.460657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:35.460657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 481/955] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:35.465603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:35.541622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:35.580048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74314225-ef71-4467-9e9a-642db0732be2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:35.617943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:35.618572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:35.618847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:35.619148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:35.619888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:35.620221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:35.621681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:35.633991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 475/956] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:35.640265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:35.745421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ee37e-8b6d-48e0-a3a5-a86972daae34 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:35.780976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:35.781609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:35.781879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:35.782143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:35.790301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:35.791823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:53:35.803784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:35.804938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:35.806109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 482/957] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:35.811757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:35.923326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd06ed1-c7d3-49b9-a214-466832dc8569 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:35.950787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:35.960269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:35.960880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:35.961150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:35.961413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:35.962139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:35.962435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:35.963928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:35.983275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 476/958] 10.222.0.36 () {56 vars in 1143 bytes} [Mon Oct 17 06:53:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:35.989071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:36.005181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:36.091894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c28b4-8c57-4c7f-b41f-2c16eded5225 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:36.127273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:36.127874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:36.128151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:36.128419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:36.199267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:36.200712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:36.214631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:36.299152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 483/959] 10.222.0.36 () {58 vars in 1282 bytes} [Mon Oct 17 06:53:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:53:36.304919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:36.402703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba10844-54ca-4875-88fd-abb23137e04c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:36.435329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:36.436543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:36.436820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:36.437086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:36.446716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:36.448168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:53:36.462049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:36.463128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:53:36.464345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:53:36.464345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:53:36.464345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:36.464345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 477/960] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:36.468837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:36.563292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8150b5f-5be3-4a23-a71d-21a76fa39ed5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:36.576034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:36.597448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:36.598040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:36.598320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:36.598591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:36.599231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:36.599540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:36.600892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:36.612236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 484/961] 10.222.0.36 () {56 vars in 1109 bytes} [Mon Oct 17 06:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:36.617709 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:36.651216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:36.712914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20986d-af31-44a9-af52-8180b0840745 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:36.745313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:36.745910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:36.746182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:36.746445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:36.752804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:36.754218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:53:36.764924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:36.765961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Oct 17 06:53:36.767131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 478/962] 10.222.0.36 () {56 vars in 1111 bytes} [Mon Oct 17 06:53:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:36.771813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:36.873772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655494d3-070c-426c-9737-b1b1b447df8d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:36.908185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec626041-3d46-4e63-9a37-208fc33f5466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:36.908771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626041-3d46-4e63-9a37-208fc33f5466 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:36.909045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626041-3d46-4e63-9a37-208fc33f5466 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:36.909310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626041-3d46-4e63-9a37-208fc33f5466 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:36.909999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec626041-3d46-4e63-9a37-208fc33f5466 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:36.910293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec626041-3d46-4e63-9a37-208fc33f5466 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:36.911733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec626041-3d46-4e63-9a37-208fc33f5466 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:36.923029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 485/963] 10.222.0.36 () {56 vars in 1148 bytes} [Mon Oct 17 06:53:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:53:36.929098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:37.019826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:37.025642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334818de-63d0-4546-98c4-79a3ab7f2185 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:37.057636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedf670-9062-4814-9aa2-6589be8b966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:37.058218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf670-9062-4814-9aa2-6589be8b966e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:37.058741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf670-9062-4814-9aa2-6589be8b966e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:37.059005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf670-9062-4814-9aa2-6589be8b966e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:37.059742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedf670-9062-4814-9aa2-6589be8b966e None admin] RBAC: Adding query filter params (role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9, scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc, user.id=522ff98d9ecb4d90bd97a19f71a400c8) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:37.060046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedf670-9062-4814-9aa2-6589be8b966e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:37.062052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedf670-9062-4814-9aa2-6589be8b966e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:37.067569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 479/964] 10.222.0.36 () {56 vars in 1374 bytes} [Mon Oct 17 06:53:36 2022] GET /identity/v3/role_assignments?user.id=522ff98d9ecb4d90bd97a19f71a400c8&scope.project.id=c1f20ff4c5f24755b1f918e07e8b23fc&role.id=cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:39.195491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135514-de8f-4de9-b764-879fb2cb9f5f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:39.195966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135514-de8f-4de9-b764-879fb2cb9f5f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:39.196056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135514-de8f-4de9-b764-879fb2cb9f5f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:39.197391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 486/965] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:39.203115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe024-7e38-4dd2-a8a1-3c7daeaba542 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:39.203585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe024-7e38-4dd2-a8a1-3c7daeaba542 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:39.203679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe024-7e38-4dd2-a8a1-3c7daeaba542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:39.221096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-69dbe024-7e38-4dd2-a8a1-3c7daeaba542 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:39.222254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-69dbe024-7e38-4dd2-a8a1-3c7daeaba542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:39.285668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 480/966] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:39.551901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00135514-de8f-4de9-b764-879fb2cb9f5f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:39.582525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:39.651750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00135514-de8f-4de9-b764-879fb2cb9f5f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:39.684491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:39.685087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:39.685361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:39.686154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:39.686773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:39.688149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:39.696565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:39.697836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:39.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 487/967] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:53:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:39.704102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe024-7e38-4dd2-a8a1-3c7daeaba542 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:39.794284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:39.803889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe024-7e38-4dd2-a8a1-3c7daeaba542 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:39.838341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c8702-f7b2-46eb-a7a1-427ea3e27020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:39.839068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8702-f7b2-46eb-a7a1-427ea3e27020 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:39.839427 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8702-f7b2-46eb-a7a1-427ea3e27020 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:39.839762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8702-f7b2-46eb-a7a1-427ea3e27020 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:39.840587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c8702-f7b2-46eb-a7a1-427ea3e27020 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:39.840991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c8702-f7b2-46eb-a7a1-427ea3e27020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:39.842909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c8702-f7b2-46eb-a7a1-427ea3e27020 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:39.848361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 481/968] 10.222.0.36 () {56 vars in 1116 bytes} [Mon Oct 17 06:53:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:53:39.854887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:39.953223 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:39.959009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f5048-55f2-44b1-8763-723f5533ff4d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:39.990325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c43c2-2fbd-4442-8644-94a0fc6cf1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:39.990905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c43c2-2fbd-4442-8644-94a0fc6cf1fe None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:39.991175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c43c2-2fbd-4442-8644-94a0fc6cf1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:39.991466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c43c2-2fbd-4442-8644-94a0fc6cf1fe None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:39.992068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c43c2-2fbd-4442-8644-94a0fc6cf1fe None admin] RBAC: Adding query filter params (type=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:39.992360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c43c2-2fbd-4442-8644-94a0fc6cf1fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:39.993708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c43c2-2fbd-4442-8644-94a0fc6cf1fe None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:39.997889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 488/969] 10.222.0.36 () {56 vars in 1116 bytes} [Mon Oct 17 06:53:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:42.114552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a4f0-4fab-4d8e-aa12-2fc6ae77667b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:42.115081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a4f0-4fab-4d8e-aa12-2fc6ae77667b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:42.115081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a4f0-4fab-4d8e-aa12-2fc6ae77667b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:42.116456 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 482/970] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:42.121516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06db2-f442-4650-bba0-8638d2e05846 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:42.121905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06db2-f442-4650-bba0-8638d2e05846 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:42.122053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06db2-f442-4650-bba0-8638d2e05846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:42.139959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b9d06db2-f442-4650-bba0-8638d2e05846 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:42.141123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b9d06db2-f442-4650-bba0-8638d2e05846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:42.205868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 489/971] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:42.472974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a4f0-4fab-4d8e-aa12-2fc6ae77667b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:42.569260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a4f0-4fab-4d8e-aa12-2fc6ae77667b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:42.601245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862b4a9-9ea4-4456-8766-df2e905a6789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:42.601825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b4a9-9ea4-4456-8766-df2e905a6789 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:42.602100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b4a9-9ea4-4456-8766-df2e905a6789 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:42.602368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b4a9-9ea4-4456-8766-df2e905a6789 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:42.602996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862b4a9-9ea4-4456-8766-df2e905a6789 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:42.604402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862b4a9-9ea4-4456-8766-df2e905a6789 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:42.618039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 483/972] 10.222.0.36 () {60 vars in 1141 bytes} [Mon Oct 17 06:53:42 2022] POST /identity/v3/services => generated 246 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:53:44.750776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0387-cc06-48be-8f52-eb09ada9944e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:44.751330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0387-cc06-48be-8f52-eb09ada9944e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:44.751330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0387-cc06-48be-8f52-eb09ada9944e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:44.752666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 490/973] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:44.758208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6087-96bd-4bec-9004-8f41f7c6eaa0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:44.758631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6087-96bd-4bec-9004-8f41f7c6eaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:44.758757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6087-96bd-4bec-9004-8f41f7c6eaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:44.776759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ca7d6087-96bd-4bec-9004-8f41f7c6eaa0 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:44.777926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ca7d6087-96bd-4bec-9004-8f41f7c6eaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:44.787414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:44.851961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 484/974] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:44 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:45.126632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e0387-cc06-48be-8f52-eb09ada9944e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:45.215749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:45.225969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e0387-cc06-48be-8f52-eb09ada9944e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:45.258737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:45.259374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:45.259654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:45.259923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:45.260488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:45.261834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:45.270205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:45.271375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:45.272394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 491/975] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:53:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:45.277239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6087-96bd-4bec-9004-8f41f7c6eaa0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:45.371989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6087-96bd-4bec-9004-8f41f7c6eaa0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:45.404386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:45.404978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:45.405254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:45.405524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:45.406133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:45.406433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:45.407820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:45.412728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 485/976] 10.222.0.36 () {56 vars in 1116 bytes} [Mon Oct 17 06:53:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:45.418476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:45.513620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12361f5b-b172-486b-b8f7-ab92d9aa9ce2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:45.545271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf6620-f791-4067-85bc-2f78bf08a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:45.545850 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf6620-f791-4067-85bc-2f78bf08a571 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:45.546125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf6620-f791-4067-85bc-2f78bf08a571 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:45.546391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf6620-f791-4067-85bc-2f78bf08a571 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:45.547094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf6620-f791-4067-85bc-2f78bf08a571 None admin] RBAC: Adding query filter params (interface=public, service_id=d1e0e4dcf0bc4052a1e5df72989e0817, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:45.547487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf6620-f791-4067-85bc-2f78bf08a571 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:45.548822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf6620-f791-4067-85bc-2f78bf08a571 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:45.553572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 492/977] 10.222.0.36 () {56 vars in 1251 bytes} [Mon Oct 17 06:53:45 2022] GET /identity/v3/endpoints?service_id=d1e0e4dcf0bc4052a1e5df72989e0817&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:45.559079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:45.563277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:45.655849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c2b43-8d83-480d-9efc-fc3661b86e6d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:45.688172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2917aab-a355-4b5d-b8e8-2965922f760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:45.688755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2917aab-a355-4b5d-b8e8-2965922f760c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:45.689031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2917aab-a355-4b5d-b8e8-2965922f760c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:45.689303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2917aab-a355-4b5d-b8e8-2965922f760c None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:45.689890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2917aab-a355-4b5d-b8e8-2965922f760c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:45.691272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2917aab-a355-4b5d-b8e8-2965922f760c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:45.697793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 486/978] 10.222.0.36 () {56 vars in 1087 bytes} [Mon Oct 17 06:53:45 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:53:47.807509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dcadf-cc4f-446a-9fb0-7f838630ce3a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:47.808353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dcadf-cc4f-446a-9fb0-7f838630ce3a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:47.808353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dcadf-cc4f-446a-9fb0-7f838630ce3a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:47.809610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 493/979] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:53:47.815165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f7b3f-d58f-498f-b9c2-5116b7cba540 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:47.815577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f7b3f-d58f-498f-b9c2-5116b7cba540 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:47.815747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f7b3f-d58f-498f-b9c2-5116b7cba540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:47.834413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-097f7b3f-d58f-498f-b9c2-5116b7cba540 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:53:47.836117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-097f7b3f-d58f-498f-b9c2-5116b7cba540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:53:47.905463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:47.911364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 487/980] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:53:47 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:53:48.179230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dcadf-cc4f-446a-9fb0-7f838630ce3a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:48.198183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:48.275659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dcadf-cc4f-446a-9fb0-7f838630ce3a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:48.307776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:48.308624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:48.308894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:48.309155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:48.309708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:48.311070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:53:48.319282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:53:48.320552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Oct 17 06:53:48.321718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 494/981] 10.222.0.36 () {56 vars in 1117 bytes} [Mon Oct 17 06:53:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:53:48.329238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f7b3f-d58f-498f-b9c2-5116b7cba540 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:48.422827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f7b3f-d58f-498f-b9c2-5116b7cba540 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:48.455109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495cba6-d6dd-4268-8a46-6387ff4f6b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:48.455718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495cba6-d6dd-4268-8a46-6387ff4f6b4c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:48.455989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495cba6-d6dd-4268-8a46-6387ff4f6b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:48.456254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495cba6-d6dd-4268-8a46-6387ff4f6b4c None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:48.456867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495cba6-d6dd-4268-8a46-6387ff4f6b4c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:53:48.457163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495cba6-d6dd-4268-8a46-6387ff4f6b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:48.458511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495cba6-d6dd-4268-8a46-6387ff4f6b4c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:48.463406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 488/982] 10.222.0.36 () {56 vars in 1116 bytes} [Mon Oct 17 06:53:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:53:48.469512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:53:48.547136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:48.565660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179afc-41a4-4c88-b976-52025440113c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:53:48.597380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a0e5c-d86d-44f1-94f4-f61463f14a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:53:48.597960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a0e5c-d86d-44f1-94f4-f61463f14a6e None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:53:48.598235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a0e5c-d86d-44f1-94f4-f61463f14a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:53:48.598504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a0e5c-d86d-44f1-94f4-f61463f14a6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:53:48.599158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a0e5c-d86d-44f1-94f4-f61463f14a6e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:53:48.600525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a0e5c-d86d-44f1-94f4-f61463f14a6e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:53:48.618928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 495/983] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:53:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:53:58.461002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=69 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:53:58.614785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:54:51.391413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b538cd-45b1-4b8f-8538-5c4742bc8f45 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:51.391900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b538cd-45b1-4b8f-8538-5c4742bc8f45 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:51.391982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b538cd-45b1-4b8f-8538-5c4742bc8f45 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:51.398482 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68277c1-2575-4e18-bdab-a05673ab6da4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:51.398889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68277c1-2575-4e18-bdab-a05673ab6da4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:51.399020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68277c1-2575-4e18-bdab-a05673ab6da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:51.401674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 489/984] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:54:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:54:51.551001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a68277c1-2575-4e18-bdab-a05673ab6da4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:54:51.552242 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a68277c1-2575-4e18-bdab-a05673ab6da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:51.678668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 496/985] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:54:51.970068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa595a5-eda1-43cc-a37c-cca111e0ef95 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:51.970346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa595a5-eda1-43cc-a37c-cca111e0ef95 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:51.970881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa595a5-eda1-43cc-a37c-cca111e0ef95 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:51.972340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 490/986] 10.222.0.36 () {54 vars in 964 bytes} [Mon Oct 17 06:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:54:51.978721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506ff63-3846-496a-8129-b2b18f277100 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:51.979012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506ff63-3846-496a-8129-b2b18f277100 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:51.979320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506ff63-3846-496a-8129-b2b18f277100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:52.073177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e506ff63-3846-496a-8129-b2b18f277100 None None] MFA Rules not processed for user `56967a5531ea4e308af5b89c13524c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:54:52.074501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e506ff63-3846-496a-8129-b2b18f277100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:52.098709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:54:52.169876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 497/987] 10.222.0.36 () {58 vars in 1061 bytes} [Mon Oct 17 06:54:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:54:52.179064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa595a5-eda1-43cc-a37c-cca111e0ef95 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:54:52.180965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eaa595a5-eda1-43cc-a37c-cca111e0ef95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:52.294360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa595a5-eda1-43cc-a37c-cca111e0ef95 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:54:52.370479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3922cd5-3a34-4533-be71-ea8c82d69c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:54:52.371082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3922cd5-3a34-4533-be71-ea8c82d69c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:52.371423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3922cd5-3a34-4533-be71-ea8c82d69c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:52.371658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3922cd5-3a34-4533-be71-ea8c82d69c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:52.373264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a3922cd5-3a34-4533-be71-ea8c82d69c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:52.431015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:54:52.489670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3922cd5-3a34-4533-be71-ea8c82d69c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:54:52.491115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3922cd5-3a34-4533-be71-ea8c82d69c55 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:54:52.511187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 491/988] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:54:54.945782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00b576-52db-4e30-a454-c6e05399a9b4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:54.946308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00b576-52db-4e30-a454-c6e05399a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:54.946308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00b576-52db-4e30-a454-c6e05399a9b4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:54.948266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 498/989] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:54:54.953640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c7728-4f07-4dfa-8c94-b762e742e6d8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:54.954025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c7728-4f07-4dfa-8c94-b762e742e6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:54.954125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c7728-4f07-4dfa-8c94-b762e742e6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:55.028984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7b3c7728-4f07-4dfa-8c94-b762e742e6d8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:54:55.030194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7b3c7728-4f07-4dfa-8c94-b762e742e6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:55.125205 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 492/990] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:54:55.405171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00b576-52db-4e30-a454-c6e05399a9b4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:54:55.406689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-de00b576-52db-4e30-a454-c6e05399a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:55.513096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00b576-52db-4e30-a454-c6e05399a9b4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:54:55.584038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:54:55.586607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162a10d-0595-45f0-90f0-206f08afb2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:54:55.587921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162a10d-0595-45f0-90f0-206f08afb2cd service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:55.588483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162a10d-0595-45f0-90f0-206f08afb2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:55.588811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162a10d-0595-45f0-90f0-206f08afb2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:55.591241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d162a10d-0595-45f0-90f0-206f08afb2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:55.698924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162a10d-0595-45f0-90f0-206f08afb2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:54:55.700395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162a10d-0595-45f0-90f0-206f08afb2cd service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:54:55.719078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 499/991] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:54:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:54:58.133339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3400-2c0c-4689-bd90-9bac62f0ecea None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:58.134002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3400-2c0c-4689-bd90-9bac62f0ecea None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:58.134002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3400-2c0c-4689-bd90-9bac62f0ecea None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:58.135266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 493/992] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:54:58.140550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083ef7-5d30-4ac3-9bbd-8b65a9f84319 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:58.140982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083ef7-5d30-4ac3-9bbd-8b65a9f84319 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:58.141099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083ef7-5d30-4ac3-9bbd-8b65a9f84319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:58.206097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f8083ef7-5d30-4ac3-9bbd-8b65a9f84319 None None] MFA Rules not processed for user `49f890768f9c4e84be27c82316e89a39`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:54:58.207325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f8083ef7-5d30-4ac3-9bbd-8b65a9f84319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:58.259533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:54:58.302550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 500/993] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:54:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:54:58.588285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3400-2c0c-4689-bd90-9bac62f0ecea None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:54:58.589825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-48dd3400-2c0c-4689-bd90-9bac62f0ecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:58.600299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:54:58.690022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3400-2c0c-4689-bd90-9bac62f0ecea None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:54:58.760633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056256de-3a4d-4a26-b221-0ee341a613e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:54:58.761219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056256de-3a4d-4a26-b221-0ee341a613e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:54:58.761489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056256de-3a4d-4a26-b221-0ee341a613e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:54:58.761759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056256de-3a4d-4a26-b221-0ee341a613e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:54:58.763326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-056256de-3a4d-4a26-b221-0ee341a613e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:54:58.823438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:54:58.869540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056256de-3a4d-4a26-b221-0ee341a613e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:54:58.870899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056256de-3a4d-4a26-b221-0ee341a613e4 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:54:58.890274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 494/994] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:54:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:01.536722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a85b7-5596-4f3e-a9b9-c7e8c6e93d53 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:01.537235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a85b7-5596-4f3e-a9b9-c7e8c6e93d53 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:01.537235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a85b7-5596-4f3e-a9b9-c7e8c6e93d53 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:01.538499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 501/995] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:01.543745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4d51-fab3-418a-bc22-049d24002977 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:01.544176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4d51-fab3-418a-bc22-049d24002977 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:01.544271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4d51-fab3-418a-bc22-049d24002977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:01.608493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2afe4d51-fab3-418a-bc22-049d24002977 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:01.609672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2afe4d51-fab3-418a-bc22-049d24002977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:01.703325 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 495/996] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:01.986964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a85b7-5596-4f3e-a9b9-c7e8c6e93d53 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:01.988520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-341a85b7-5596-4f3e-a9b9-c7e8c6e93d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:02.088416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a85b7-5596-4f3e-a9b9-c7e8c6e93d53 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:02.159705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d0a87-962f-4d73-b335-e5f2db4745ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:02.160380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0a87-962f-4d73-b335-e5f2db4745ba service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:02.160579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0a87-962f-4d73-b335-e5f2db4745ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:02.160845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0a87-962f-4d73-b335-e5f2db4745ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:02.162403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8d2d0a87-962f-4d73-b335-e5f2db4745ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:02.169777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:02.267757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d0a87-962f-4d73-b335-e5f2db4745ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:02.269102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d0a87-962f-4d73-b335-e5f2db4745ba service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:02.286789 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 502/997] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:05.049034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebed36-cb3a-48fc-8f74-5223cfba069d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:05.049571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebed36-cb3a-48fc-8f74-5223cfba069d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:05.049571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebed36-cb3a-48fc-8f74-5223cfba069d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:05.050936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 496/998] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:05.057105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248918-8429-4ee5-ac8d-eac2bc585d20 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:05.057531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248918-8429-4ee5-ac8d-eac2bc585d20 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:05.057638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248918-8429-4ee5-ac8d-eac2bc585d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:05.132461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d2248918-8429-4ee5-ac8d-eac2bc585d20 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:05.133755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d2248918-8429-4ee5-ac8d-eac2bc585d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:05.196214 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:05.238910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 503/999] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:55:05.521240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebed36-cb3a-48fc-8f74-5223cfba069d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:05.522758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-39ebed36-cb3a-48fc-8f74-5223cfba069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:05.542832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:05.626480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebed36-cb3a-48fc-8f74-5223cfba069d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:05.699130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd5f29-9dfe-4657-a6de-a5ced45625a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:05.699761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd5f29-9dfe-4657-a6de-a5ced45625a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:05.700038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd5f29-9dfe-4657-a6de-a5ced45625a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:05.700313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd5f29-9dfe-4657-a6de-a5ced45625a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:05.701887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f0dd5f29-9dfe-4657-a6de-a5ced45625a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:05.772940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:05.811089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd5f29-9dfe-4657-a6de-a5ced45625a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:05.812562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd5f29-9dfe-4657-a6de-a5ced45625a0 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:05.830064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 497/1000] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:08.028706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b491c-3d3f-4875-81b9-98562a4491c9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:08.029230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b491c-3d3f-4875-81b9-98562a4491c9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:08.029230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b491c-3d3f-4875-81b9-98562a4491c9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:08.030583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 504/1001] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:08.035875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349dd7-f5fc-4186-b225-dc28c9d8e063 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:08.036294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349dd7-f5fc-4186-b225-dc28c9d8e063 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:08.036448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349dd7-f5fc-4186-b225-dc28c9d8e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:08.103652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b4349dd7-f5fc-4186-b225-dc28c9d8e063 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:08.104848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b4349dd7-f5fc-4186-b225-dc28c9d8e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:08.207408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 498/1002] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:08.490082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b491c-3d3f-4875-81b9-98562a4491c9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:08.491674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-509b491c-3d3f-4875-81b9-98562a4491c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:08.594010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b491c-3d3f-4875-81b9-98562a4491c9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:08.666589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe57a2-9ef4-4ca4-aead-741caaccaab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:08.667193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe57a2-9ef4-4ca4-aead-741caaccaab1 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:08.667499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe57a2-9ef4-4ca4-aead-741caaccaab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:08.667771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe57a2-9ef4-4ca4-aead-741caaccaab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:08.669347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fbfe57a2-9ef4-4ca4-aead-741caaccaab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:08.686372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:08.781686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe57a2-9ef4-4ca4-aead-741caaccaab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:08.783029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe57a2-9ef4-4ca4-aead-741caaccaab1 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:08.801297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 505/1003] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:10.983521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fa4ea-195f-4bf1-87f1-03eee0868f67 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:10.984052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fa4ea-195f-4bf1-87f1-03eee0868f67 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:10.984052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fa4ea-195f-4bf1-87f1-03eee0868f67 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:10.985372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 499/1004] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:10.990455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dda73-c5d9-44ea-8818-59526b4547ef None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:10.990826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dda73-c5d9-44ea-8818-59526b4547ef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:10.990998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dda73-c5d9-44ea-8818-59526b4547ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:11.056800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a44dda73-c5d9-44ea-8818-59526b4547ef None None] MFA Rules not processed for user `49f890768f9c4e84be27c82316e89a39`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:11.057917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a44dda73-c5d9-44ea-8818-59526b4547ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:11.121137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:11.153748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 506/1005] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:11.434027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fa4ea-195f-4bf1-87f1-03eee0868f67 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:11.435599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c05fa4ea-195f-4bf1-87f1-03eee0868f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:11.468682 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:11.543627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fa4ea-195f-4bf1-87f1-03eee0868f67 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:11.618004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35ebc2-778a-4dfd-ab93-eb9d356083c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:11.618621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35ebc2-778a-4dfd-ab93-eb9d356083c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:11.618894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35ebc2-778a-4dfd-ab93-eb9d356083c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:11.619159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35ebc2-778a-4dfd-ab93-eb9d356083c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:11.620736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9b35ebc2-778a-4dfd-ab93-eb9d356083c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:11.698006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:11.729970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35ebc2-778a-4dfd-ab93-eb9d356083c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:11.731438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35ebc2-778a-4dfd-ab93-eb9d356083c7 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:11.749124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 500/1006] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:14.122093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13dfa5-0473-4dc5-a56d-cb10826e1c86 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:14.122619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13dfa5-0473-4dc5-a56d-cb10826e1c86 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:14.122619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13dfa5-0473-4dc5-a56d-cb10826e1c86 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:14.123999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 507/1007] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:14.129284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734eaff-758e-44e9-8a6d-8f697b694321 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:14.129694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734eaff-758e-44e9-8a6d-8f697b694321 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:14.129831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734eaff-758e-44e9-8a6d-8f697b694321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:14.196657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b734eaff-758e-44e9-8a6d-8f697b694321 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:14.197853 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b734eaff-758e-44e9-8a6d-8f697b694321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:14.290305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 501/1008] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:14.567592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13dfa5-0473-4dc5-a56d-cb10826e1c86 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:14.569143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8b13dfa5-0473-4dc5-a56d-cb10826e1c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:14.670160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13dfa5-0473-4dc5-a56d-cb10826e1c86 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:14.742384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3240a6-342e-4941-a99a-5e5991a76ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:14.743019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3240a6-342e-4941-a99a-5e5991a76ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:14.743324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3240a6-342e-4941-a99a-5e5991a76ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:14.743599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3240a6-342e-4941-a99a-5e5991a76ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:14.745180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eb3240a6-342e-4941-a99a-5e5991a76ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:14.771770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:14.852652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3240a6-342e-4941-a99a-5e5991a76ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:14.854008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3240a6-342e-4941-a99a-5e5991a76ca4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:14.871927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 508/1009] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:17.040862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670146d6-cbde-4034-a3e8-44917d25053c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:17.041380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670146d6-cbde-4034-a3e8-44917d25053c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:17.041380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670146d6-cbde-4034-a3e8-44917d25053c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:17.042650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 502/1010] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:17.047972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0f8f-48f5-4639-9a8f-6bb0a12371c8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:17.048388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0f8f-48f5-4639-9a8f-6bb0a12371c8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:17.048510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0f8f-48f5-4639-9a8f-6bb0a12371c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:17.113268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d77a0f8f-48f5-4639-9a8f-6bb0a12371c8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:17.114454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d77a0f8f-48f5-4639-9a8f-6bb0a12371c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:17.187404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:17.208995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 509/1011] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:17.491738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670146d6-cbde-4034-a3e8-44917d25053c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:17.493259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-670146d6-cbde-4034-a3e8-44917d25053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:17.533260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:17.596223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670146d6-cbde-4034-a3e8-44917d25053c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:17.667612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c97eb8-0159-4a97-96dc-d89c97fb5097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:17.668273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c97eb8-0159-4a97-96dc-d89c97fb5097 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:17.668554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c97eb8-0159-4a97-96dc-d89c97fb5097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:17.668825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c97eb8-0159-4a97-96dc-d89c97fb5097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:17.670406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-70c97eb8-0159-4a97-96dc-d89c97fb5097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:17.750912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:17.777376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c97eb8-0159-4a97-96dc-d89c97fb5097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:17.778810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c97eb8-0159-4a97-96dc-d89c97fb5097 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:17.796106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 503/1012] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:20.228445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c2bbd-304f-48e7-a707-444097811cb9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:20.228989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c2bbd-304f-48e7-a707-444097811cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:20.228989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c2bbd-304f-48e7-a707-444097811cb9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:20.230300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 510/1013] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:20.235521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedca7e-a1e5-45c2-83e2-675c1c618dd2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:20.235957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedca7e-a1e5-45c2-83e2-675c1c618dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:20.236063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedca7e-a1e5-45c2-83e2-675c1c618dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:20.300780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0eedca7e-a1e5-45c2-83e2-675c1c618dd2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:20.301961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0eedca7e-a1e5-45c2-83e2-675c1c618dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:20.395404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 504/1014] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:20.674535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c2bbd-304f-48e7-a707-444097811cb9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:20.676121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f63c2bbd-304f-48e7-a707-444097811cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:20.777636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c2bbd-304f-48e7-a707-444097811cb9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:20.852667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f054eb5-01fe-4f71-8b88-597b9743a590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:20.853420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f054eb5-01fe-4f71-8b88-597b9743a590 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:20.853842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f054eb5-01fe-4f71-8b88-597b9743a590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:20.854269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f054eb5-01fe-4f71-8b88-597b9743a590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:20.856876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3f054eb5-01fe-4f71-8b88-597b9743a590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:20.893893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:20.965992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f054eb5-01fe-4f71-8b88-597b9743a590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:20.967441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f054eb5-01fe-4f71-8b88-597b9743a590 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:20.984827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 511/1015] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:24.997626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c8ef-2084-422a-b377-27496a905da2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:24.998135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c8ef-2084-422a-b377-27496a905da2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:24.998135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c8ef-2084-422a-b377-27496a905da2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:24.999639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 505/1016] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:25.004649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3bd17-563b-4c8b-a318-502c75105445 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:25.005032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3bd17-563b-4c8b-a318-502c75105445 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:25.005189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3bd17-563b-4c8b-a318-502c75105445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:25.071183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f0a3bd17-563b-4c8b-a318-502c75105445 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:25.072310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f0a3bd17-563b-4c8b-a318-502c75105445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:25.157078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:25.168344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 512/1017] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:25.452508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c8ef-2084-422a-b377-27496a905da2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:25.454087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6778c8ef-2084-422a-b377-27496a905da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:25.502272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:25.557450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c8ef-2084-422a-b377-27496a905da2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:25.631137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9018c5-372f-475f-b7d5-0e36b46a090a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:25.631763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9018c5-372f-475f-b7d5-0e36b46a090a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:25.632043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9018c5-372f-475f-b7d5-0e36b46a090a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:25.632314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9018c5-372f-475f-b7d5-0e36b46a090a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:25.633889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-db9018c5-372f-475f-b7d5-0e36b46a090a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:25.725087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:25.742734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9018c5-372f-475f-b7d5-0e36b46a090a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:25.744203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9018c5-372f-475f-b7d5-0e36b46a090a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:25.761826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 506/1018] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:28.559924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38045cb7-9c56-4db3-b94a-37400565aa3e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:28.560492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38045cb7-9c56-4db3-b94a-37400565aa3e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:28.560492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38045cb7-9c56-4db3-b94a-37400565aa3e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:28.561958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 513/1019] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:28.566889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95ef8e-c2b8-4b38-8ce5-d6192629ac32 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:28.567285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95ef8e-c2b8-4b38-8ce5-d6192629ac32 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:28.567472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95ef8e-c2b8-4b38-8ce5-d6192629ac32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:28.632374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4c95ef8e-c2b8-4b38-8ce5-d6192629ac32 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:28.633579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4c95ef8e-c2b8-4b38-8ce5-d6192629ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:28.732241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 507/1020] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:29.031456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86073-f4a6-47fc-8617-495db0d4e85c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:29.031834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86073-f4a6-47fc-8617-495db0d4e85c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:29.032196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86073-f4a6-47fc-8617-495db0d4e85c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:29.033612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 514/1021] 10.222.0.36 () {54 vars in 964 bytes} [Mon Oct 17 06:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:29.040356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcc53f-89a6-42ee-ae09-2347c3769263 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:29.040645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcc53f-89a6-42ee-ae09-2347c3769263 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:29.040925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcc53f-89a6-42ee-ae09-2347c3769263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:29.112353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:29.121711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5ebcc53f-89a6-42ee-ae09-2347c3769263 None None] MFA Rules not processed for user `56967a5531ea4e308af5b89c13524c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:29.122911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5ebcc53f-89a6-42ee-ae09-2347c3769263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:29.204799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 508/1022] 10.222.0.36 () {58 vars in 1061 bytes} [Mon Oct 17 06:55:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:29.214114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa86073-f4a6-47fc-8617-495db0d4e85c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:29.215651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3fa86073-f4a6-47fc-8617-495db0d4e85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:29.320945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa86073-f4a6-47fc-8617-495db0d4e85c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:29.395566 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa454a1-df2a-41a2-8476-f691029e3928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:29.396160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa454a1-df2a-41a2-8476-f691029e3928 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:29.396435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa454a1-df2a-41a2-8476-f691029e3928 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:29.396698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa454a1-df2a-41a2-8476-f691029e3928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:29.398785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2aa454a1-df2a-41a2-8476-f691029e3928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:29.457011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:29.529130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa454a1-df2a-41a2-8476-f691029e3928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:29.530574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa454a1-df2a-41a2-8476-f691029e3928 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:29.551253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 515/1023] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:34.388448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5cf9d-ec2d-4ec7-a85f-41a02793450c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:34.388981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5cf9d-ec2d-4ec7-a85f-41a02793450c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:34.388981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5cf9d-ec2d-4ec7-a85f-41a02793450c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:34.390317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 509/1024] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:34.395579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bce2d7-9cc0-4c15-be1f-c5bf6a353a2d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:34.395994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bce2d7-9cc0-4c15-be1f-c5bf6a353a2d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:34.396118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bce2d7-9cc0-4c15-be1f-c5bf6a353a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:34.465609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-69bce2d7-9cc0-4c15-be1f-c5bf6a353a2d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:34.466823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69bce2d7-9cc0-4c15-be1f-c5bf6a353a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:34.560731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:34.563095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 516/1025] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:34.843200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5cf9d-ec2d-4ec7-a85f-41a02793450c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:34.845465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aba5cf9d-ec2d-4ec7-a85f-41a02793450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:34.954896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5cf9d-ec2d-4ec7-a85f-41a02793450c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:35.002488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:35.027531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd838c-1174-41a7-9203-826ebb24e31f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:35.028119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd838c-1174-41a7-9203-826ebb24e31f service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:35.028389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd838c-1174-41a7-9203-826ebb24e31f service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:35.028655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd838c-1174-41a7-9203-826ebb24e31f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:35.030210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0bdd838c-1174-41a7-9203-826ebb24e31f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:35.135300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd838c-1174-41a7-9203-826ebb24e31f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:35.136658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd838c-1174-41a7-9203-826ebb24e31f service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:35.154724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 510/1026] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:40.046309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151e96b-b63a-4236-a0a7-df4e0e6daf04 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:40.046727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151e96b-b63a-4236-a0a7-df4e0e6daf04 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:40.047029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151e96b-b63a-4236-a0a7-df4e0e6daf04 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:40.050310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 517/1027] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 06:55:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:40.063863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167d65-ff0d-4220-82e9-e0b2a9373e29 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:40.064280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167d65-ff0d-4220-82e9-e0b2a9373e29 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:40.064715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167d65-ff0d-4220-82e9-e0b2a9373e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:40.207269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b3167d65-ff0d-4220-82e9-e0b2a9373e29 None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:40.207269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b3167d65-ff0d-4220-82e9-e0b2a9373e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:40.229151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:40.338492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 511/1028] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 06:55:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:44.391883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93b15f-228b-4cb2-9f5a-007e0e28e196 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:44.392862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93b15f-228b-4cb2-9f5a-007e0e28e196 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:44.393197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93b15f-228b-4cb2-9f5a-007e0e28e196 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:44.394752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 518/1029] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 06:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:44.400657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783633a-b613-4720-8209-197d3b44ce96 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:44.401082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783633a-b613-4720-8209-197d3b44ce96 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:44.401391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783633a-b613-4720-8209-197d3b44ce96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:44.481188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8783633a-b613-4720-8209-197d3b44ce96 None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:44.482729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8783633a-b613-4720-8209-197d3b44ce96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:44.561487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:44.602203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 512/1030] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 06:55:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:46.018144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642c641-1aa2-4dc5-a9f2-cfd650227716 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:46.018729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642c641-1aa2-4dc5-a9f2-cfd650227716 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:46.018729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642c641-1aa2-4dc5-a9f2-cfd650227716 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:46.020028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 519/1031] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 06:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:46.024967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986789b5-5e03-4ab4-8664-667bbb1acf01 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:46.025392 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986789b5-5e03-4ab4-8664-667bbb1acf01 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:46.025634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986789b5-5e03-4ab4-8664-667bbb1acf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:46.103458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-986789b5-5e03-4ab4-8664-667bbb1acf01 None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:46.104643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-986789b5-5e03-4ab4-8664-667bbb1acf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:46.211614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 513/1032] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 06:55:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:55:49.732111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d20a18-af2f-494a-8459-8cdf38bc0ba3 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:49.732710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d20a18-af2f-494a-8459-8cdf38bc0ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:49.732710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d20a18-af2f-494a-8459-8cdf38bc0ba3 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:49.734247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 520/1033] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:49.739736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b48b7-dcdf-4249-b7e7-e0d7ab1282b8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:49.740230 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b48b7-dcdf-4249-b7e7-e0d7ab1282b8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:49.740366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b48b7-dcdf-4249-b7e7-e0d7ab1282b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:49.778220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:49.812670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3b3b48b7-dcdf-4249-b7e7-e0d7ab1282b8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:49.813883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3b3b48b7-dcdf-4249-b7e7-e0d7ab1282b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:49.927823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 514/1034] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:50.231723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2bea-88a2-4339-aec1-9e07c00240db None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:50.232217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2bea-88a2-4339-aec1-9e07c00240db None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:50.232710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2bea-88a2-4339-aec1-9e07c00240db None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:50.234742 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 521/1035] 10.222.0.36 () {54 vars in 961 bytes} [Mon Oct 17 06:55:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:50.245274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900322a-d568-41db-adad-44766381e717 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:50.246140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900322a-d568-41db-adad-44766381e717 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:50.246884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900322a-d568-41db-adad-44766381e717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:50.335098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4900322a-d568-41db-adad-44766381e717 None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:50.336302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4900322a-d568-41db-adad-44766381e717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:50.393608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:50.464704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 515/1036] 10.222.0.36 () {58 vars in 1058 bytes} [Mon Oct 17 06:55:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:55:50.472346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a2bea-88a2-4339-aec1-9e07c00240db None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:50.473837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d62a2bea-88a2-4339-aec1-9e07c00240db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:50.629620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a2bea-88a2-4339-aec1-9e07c00240db None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:50.704923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:50.751173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5da2f7-105b-4bb8-afd8-2f57186b4af6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:50.751939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5da2f7-105b-4bb8-afd8-2f57186b4af6 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:50.752081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5da2f7-105b-4bb8-afd8-2f57186b4af6 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:50.752354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5da2f7-105b-4bb8-afd8-2f57186b4af6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:50.754026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3b5da2f7-105b-4bb8-afd8-2f57186b4af6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:50.854265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5da2f7-105b-4bb8-afd8-2f57186b4af6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:50.855954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5da2f7-105b-4bb8-afd8-2f57186b4af6 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:50.875082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 522/1037] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:53.258906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f84aa-ee06-4740-b8ac-92585ae8cafc None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:53.259630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f84aa-ee06-4740-b8ac-92585ae8cafc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:53.259630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f84aa-ee06-4740-b8ac-92585ae8cafc None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:53.260977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 516/1038] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:53.265971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0d07-a2f2-4936-9b5f-c7dbe5305e7c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:53.266511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0d07-a2f2-4936-9b5f-c7dbe5305e7c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:53.266511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0d07-a2f2-4936-9b5f-c7dbe5305e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:53.340283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b1cb0d07-a2f2-4936-9b5f-c7dbe5305e7c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:53.341484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b1cb0d07-a2f2-4936-9b5f-c7dbe5305e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:53.381911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:53.455265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 523/1039] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:53.740454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f84aa-ee06-4740-b8ac-92585ae8cafc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:53.742059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-547f84aa-ee06-4740-b8ac-92585ae8cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:53.924568 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f84aa-ee06-4740-b8ac-92585ae8cafc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:53.934789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:54.048621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43df87f-d5b5-4fc0-bb63-13c809f72efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:54.049225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df87f-d5b5-4fc0-bb63-13c809f72efa service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:54.049501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df87f-d5b5-4fc0-bb63-13c809f72efa service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:54.049769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df87f-d5b5-4fc0-bb63-13c809f72efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:54.051442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d43df87f-d5b5-4fc0-bb63-13c809f72efa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:54.144497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43df87f-d5b5-4fc0-bb63-13c809f72efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:54.146021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43df87f-d5b5-4fc0-bb63-13c809f72efa service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:54.167884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 517/1040] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:56.644949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db611-8d7f-4910-97d0-69f4a4dc6998 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:56.645533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db611-8d7f-4910-97d0-69f4a4dc6998 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:56.645533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db611-8d7f-4910-97d0-69f4a4dc6998 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:56.647167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 524/1041] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:56.652283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bf500-901f-4b7e-8680-63fcb63d866d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:56.652705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bf500-901f-4b7e-8680-63fcb63d866d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:56.652830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bf500-901f-4b7e-8680-63fcb63d866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:56.668649 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:56.721956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f04bf500-901f-4b7e-8680-63fcb63d866d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:56.723066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f04bf500-901f-4b7e-8680-63fcb63d866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:56.825206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 518/1042] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:57.127629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398138bf-de1f-4514-b4ac-09d9aff2f4a5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:57.127926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398138bf-de1f-4514-b4ac-09d9aff2f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:57.128197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398138bf-de1f-4514-b4ac-09d9aff2f4a5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:57.129552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 525/1043] 10.222.0.36 () {54 vars in 961 bytes} [Mon Oct 17 06:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:57.136568 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f9ced-6238-4fbb-bf46-a778924efabf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:57.136860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f9ced-6238-4fbb-bf46-a778924efabf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:57.137272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f9ced-6238-4fbb-bf46-a778924efabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:57.221496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3b9f9ced-6238-4fbb-bf46-a778924efabf None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:57.222696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3b9f9ced-6238-4fbb-bf46-a778924efabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:57.260115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:57.347915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 519/1044] 10.222.0.36 () {58 vars in 1058 bytes} [Mon Oct 17 06:55:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:57.357427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398138bf-de1f-4514-b4ac-09d9aff2f4a5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:57.358904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-398138bf-de1f-4514-b4ac-09d9aff2f4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:57.508865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398138bf-de1f-4514-b4ac-09d9aff2f4a5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:57.529779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:57.629784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc758c71-15cc-497b-b754-4ed3096d8f5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:57.631481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758c71-15cc-497b-b754-4ed3096d8f5d service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:57.631971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758c71-15cc-497b-b754-4ed3096d8f5d service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:57.632467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758c71-15cc-497b-b754-4ed3096d8f5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:57.634364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dc758c71-15cc-497b-b754-4ed3096d8f5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:57.741466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc758c71-15cc-497b-b754-4ed3096d8f5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:57.743996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc758c71-15cc-497b-b754-4ed3096d8f5d service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:57.759847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbaa4-b80f-477d-bfa5-465b7951ac3c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:57.760329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbaa4-b80f-477d-bfa5-465b7951ac3c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:57.760329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbaa4-b80f-477d-bfa5-465b7951ac3c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:57.761630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 520/1045] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:57.765212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 526/1046] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:55:57.766543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74d732-7bb5-4424-ba8f-5a895fee84b1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:57.766954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74d732-7bb5-4424-ba8f-5a895fee84b1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:57.767236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74d732-7bb5-4424-ba8f-5a895fee84b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:57.838792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4d74d732-7bb5-4424-ba8f-5a895fee84b1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:57.840543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4d74d732-7bb5-4424-ba8f-5a895fee84b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:57.956640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:57.966305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 521/1047] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:58.298909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b042-cba3-4ea7-946b-78007b407240 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:58.299823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b042-cba3-4ea7-946b-78007b407240 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:58.299823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b042-cba3-4ea7-946b-78007b407240 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:58.300900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 527/1048] 10.222.0.36 () {54 vars in 963 bytes} [Mon Oct 17 06:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:55:58.306542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317c39-57ec-436e-93b9-d7d6a891fbb0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:58.306888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317c39-57ec-436e-93b9-d7d6a891fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:58.307247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317c39-57ec-436e-93b9-d7d6a891fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:58.389092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-52317c39-57ec-436e-93b9-d7d6a891fbb0 None None] MFA Rules not processed for user `92c66f0a106f4c8e99ac7075d76767d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:55:58.390325 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-52317c39-57ec-436e-93b9-d7d6a891fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:58.472083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 522/1049] 10.222.0.36 () {58 vars in 1060 bytes} [Mon Oct 17 06:55:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:55:58.480156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b042-cba3-4ea7-946b-78007b407240 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:55:58.481741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9ca8b042-cba3-4ea7-946b-78007b407240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:58.511841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:58.596355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b042-cba3-4ea7-946b-78007b407240 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:55:58.681999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380cccd-99d9-4b5c-bdae-e48e4a267811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:55:58.682612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380cccd-99d9-4b5c-bdae-e48e4a267811 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:55:58.682888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380cccd-99d9-4b5c-bdae-e48e4a267811 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:55:58.683158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380cccd-99d9-4b5c-bdae-e48e4a267811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:55:58.684800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8380cccd-99d9-4b5c-bdae-e48e4a267811 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:55:58.820847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:55:58.878417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380cccd-99d9-4b5c-bdae-e48e4a267811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:55:58.880015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380cccd-99d9-4b5c-bdae-e48e4a267811 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:55:58.900660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 528/1050] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 06:56:00.002829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13829a3c-450a-4e45-be98-90548934258f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:00.003413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13829a3c-450a-4e45-be98-90548934258f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:00.003413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13829a3c-450a-4e45-be98-90548934258f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:00.004720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 523/1051] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:00.009860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2a64-251f-4152-a7a4-97eaa0cbfa71 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:00.010288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2a64-251f-4152-a7a4-97eaa0cbfa71 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:00.010402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2a64-251f-4152-a7a4-97eaa0cbfa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:00.077395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b05e2a64-251f-4152-a7a4-97eaa0cbfa71 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:00.078625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b05e2a64-251f-4152-a7a4-97eaa0cbfa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:00.178005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 529/1052] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:00.472489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13829a3c-450a-4e45-be98-90548934258f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:00.474566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-13829a3c-450a-4e45-be98-90548934258f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:00.562180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:00.625521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13829a3c-450a-4e45-be98-90548934258f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:00.742623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c20a47-9516-4a4e-b4b6-caba81a782e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:00.743356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20a47-9516-4a4e-b4b6-caba81a782e6 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:00.743522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20a47-9516-4a4e-b4b6-caba81a782e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:00.744092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20a47-9516-4a4e-b4b6-caba81a782e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:00.745465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-60c20a47-9516-4a4e-b4b6-caba81a782e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:00.796903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:00.841521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c20a47-9516-4a4e-b4b6-caba81a782e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:00.843888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c20a47-9516-4a4e-b4b6-caba81a782e6 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:00.864000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 524/1053] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 06:56:03.087312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccbbca-2296-4145-bc23-9c35583659ca None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:03.087840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccbbca-2296-4145-bc23-9c35583659ca None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:03.087840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccbbca-2296-4145-bc23-9c35583659ca None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:03.089185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 530/1054] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:03.094198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f88c1-b3ff-4d6b-93e2-2d6553f998c3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:03.094586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f88c1-b3ff-4d6b-93e2-2d6553f998c3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:03.094778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f88c1-b3ff-4d6b-93e2-2d6553f998c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:03.177519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-360f88c1-b3ff-4d6b-93e2-2d6553f998c3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:03.179041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-360f88c1-b3ff-4d6b-93e2-2d6553f998c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:03.284073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 525/1055] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:03.573582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccbbca-2296-4145-bc23-9c35583659ca None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:03.575385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d0ccbbca-2296-4145-bc23-9c35583659ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:03.617775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:03.731236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccbbca-2296-4145-bc23-9c35583659ca None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:03.844106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:03.852857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a612e71-4e9e-4efe-90cc-c7792fad56be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:03.853527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612e71-4e9e-4efe-90cc-c7792fad56be service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:03.853800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612e71-4e9e-4efe-90cc-c7792fad56be service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:03.854065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612e71-4e9e-4efe-90cc-c7792fad56be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:03.855792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0a612e71-4e9e-4efe-90cc-c7792fad56be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:03.952196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a612e71-4e9e-4efe-90cc-c7792fad56be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:03.953801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a612e71-4e9e-4efe-90cc-c7792fad56be service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:03.972926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 531/1056] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:06.170654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace154e-5ac5-4627-b305-b11d47a15b93 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:06.171104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace154e-5ac5-4627-b305-b11d47a15b93 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:06.171199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace154e-5ac5-4627-b305-b11d47a15b93 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:06.172553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 526/1057] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:06.178844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70803536-d361-4c5c-8e23-486e3ab84b06 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:06.179293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70803536-d361-4c5c-8e23-486e3ab84b06 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:06.179520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70803536-d361-4c5c-8e23-486e3ab84b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:06.287503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-70803536-d361-4c5c-8e23-486e3ab84b06 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:06.289923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-70803536-d361-4c5c-8e23-486e3ab84b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:06.363502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:06.397811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 532/1058] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:06.691388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace154e-5ac5-4627-b305-b11d47a15b93 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:06.692965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bace154e-5ac5-4627-b305-b11d47a15b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:06.722274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:06.845417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace154e-5ac5-4627-b305-b11d47a15b93 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:06.945717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:06.962278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3eea7-58e2-4779-b215-89e0e9edaeec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:06.962916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3eea7-58e2-4779-b215-89e0e9edaeec service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:06.963191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3eea7-58e2-4779-b215-89e0e9edaeec service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:06.963491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3eea7-58e2-4779-b215-89e0e9edaeec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:06.965339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eea3eea7-58e2-4779-b215-89e0e9edaeec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:07.078847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3eea7-58e2-4779-b215-89e0e9edaeec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:07.081322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3eea7-58e2-4779-b215-89e0e9edaeec service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:07.106291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 527/1059] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:10.104339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07f8d3-fce1-41e3-a2ee-a4d08a44bd4a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:10.105160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07f8d3-fce1-41e3-a2ee-a4d08a44bd4a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:10.105160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07f8d3-fce1-41e3-a2ee-a4d08a44bd4a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:10.106437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 533/1060] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:56:10.112274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae6d65-7c0b-4a93-b392-6b08e1a8cd0a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:10.112766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae6d65-7c0b-4a93-b392-6b08e1a8cd0a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:10.112861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae6d65-7c0b-4a93-b392-6b08e1a8cd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:10.206939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8cae6d65-7c0b-4a93-b392-6b08e1a8cd0a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:10.208254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8cae6d65-7c0b-4a93-b392-6b08e1a8cd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:10.272136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:10.312459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 528/1061] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:10.604092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07f8d3-fce1-41e3-a2ee-a4d08a44bd4a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:10.605618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cb07f8d3-fce1-41e3-a2ee-a4d08a44bd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:10.751090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07f8d3-fce1-41e3-a2ee-a4d08a44bd4a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:10.805659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:10.868798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc69c4-d464-44ec-9f04-b6d05dcdd379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:10.869322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc69c4-d464-44ec-9f04-b6d05dcdd379 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:10.869597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc69c4-d464-44ec-9f04-b6d05dcdd379 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:10.869871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc69c4-d464-44ec-9f04-b6d05dcdd379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:10.871479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eddc69c4-d464-44ec-9f04-b6d05dcdd379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:10.971395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc69c4-d464-44ec-9f04-b6d05dcdd379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:10.973459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc69c4-d464-44ec-9f04-b6d05dcdd379 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:10.997513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 534/1062] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:13.147414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8ae70-f310-4204-9056-bb1cdc7bbbff None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:13.147933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8ae70-f310-4204-9056-bb1cdc7bbbff None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:13.147933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8ae70-f310-4204-9056-bb1cdc7bbbff None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:13.149218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 529/1063] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:13.154771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfeee9f-5a4f-4a01-a73c-f1b88b7fd4df None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:13.155442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfeee9f-5a4f-4a01-a73c-f1b88b7fd4df None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:13.155442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfeee9f-5a4f-4a01-a73c-f1b88b7fd4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:13.226175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6dfeee9f-5a4f-4a01-a73c-f1b88b7fd4df None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:13.227455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6dfeee9f-5a4f-4a01-a73c-f1b88b7fd4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:13.236171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:13.328648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 535/1064] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:13.714156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8ae70-f310-4204-9056-bb1cdc7bbbff None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:13.715697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-59d8ae70-f310-4204-9056-bb1cdc7bbbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:13.874805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8ae70-f310-4204-9056-bb1cdc7bbbff None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:13.934261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:14.013379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977fd54-3237-4687-9d2e-53860d5492c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:14.013997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977fd54-3237-4687-9d2e-53860d5492c9 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:14.014274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977fd54-3237-4687-9d2e-53860d5492c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:14.014558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977fd54-3237-4687-9d2e-53860d5492c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:14.016235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5977fd54-3237-4687-9d2e-53860d5492c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:14.106804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977fd54-3237-4687-9d2e-53860d5492c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:14.108473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977fd54-3237-4687-9d2e-53860d5492c9 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:14.127464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 530/1065] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:16.399666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76def7e5-ac9a-4e36-ae98-30a997338a62 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:16.400258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76def7e5-ac9a-4e36-ae98-30a997338a62 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:16.400258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76def7e5-ac9a-4e36-ae98-30a997338a62 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:16.401552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 536/1066] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:16.406948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717b5fb-eee7-4fe3-9314-1a7b97cda19b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:16.407426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717b5fb-eee7-4fe3-9314-1a7b97cda19b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:16.407426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717b5fb-eee7-4fe3-9314-1a7b97cda19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:16.471471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:16.476043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c717b5fb-eee7-4fe3-9314-1a7b97cda19b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:16.477500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c717b5fb-eee7-4fe3-9314-1a7b97cda19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:16.577658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 531/1067] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:16.867427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76def7e5-ac9a-4e36-ae98-30a997338a62 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:16.868967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-76def7e5-ac9a-4e36-ae98-30a997338a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:17.009339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:17.016231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76def7e5-ac9a-4e36-ae98-30a997338a62 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:17.129220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e07e2-5efc-4aa5-a918-b50a67bf0fde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:17.129834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e07e2-5efc-4aa5-a918-b50a67bf0fde service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:17.130116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e07e2-5efc-4aa5-a918-b50a67bf0fde service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:17.130980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e07e2-5efc-4aa5-a918-b50a67bf0fde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:17.132589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-366e07e2-5efc-4aa5-a918-b50a67bf0fde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:17.224322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e07e2-5efc-4aa5-a918-b50a67bf0fde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:17.226065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e07e2-5efc-4aa5-a918-b50a67bf0fde service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:17.240181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:17.244938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 537/1068] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:17.526476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556d6e-b70b-4974-a321-def9770749b5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:17.526990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556d6e-b70b-4974-a321-def9770749b5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:17.526990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556d6e-b70b-4974-a321-def9770749b5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:17.528398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 532/1069] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:17.533681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6da2b-c941-4eed-8f53-61bdfef23f6c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:17.534123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6da2b-c941-4eed-8f53-61bdfef23f6c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:17.534226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6da2b-c941-4eed-8f53-61bdfef23f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:17.601552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4fd6da2b-c941-4eed-8f53-61bdfef23f6c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:17.602735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fd6da2b-c941-4eed-8f53-61bdfef23f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:17.701424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 538/1070] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:18.002525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb28e12-4ca3-49e5-b654-75f4fffcc105 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:18.002880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb28e12-4ca3-49e5-b654-75f4fffcc105 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:18.003255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb28e12-4ca3-49e5-b654-75f4fffcc105 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:18.004579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 533/1071] 10.222.0.36 () {54 vars in 962 bytes} [Mon Oct 17 06:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:18.010602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd8405-9fec-4b28-bb6d-a7ef9224a546 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:18.010882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd8405-9fec-4b28-bb6d-a7ef9224a546 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:18.011151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd8405-9fec-4b28-bb6d-a7ef9224a546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:18.088716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1fcd8405-9fec-4b28-bb6d-a7ef9224a546 None None] MFA Rules not processed for user `a0a3956bb98749e6bb19ee7f3465eac9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:18.089913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1fcd8405-9fec-4b28-bb6d-a7ef9224a546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:18.107914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:18.173944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 539/1072] 10.222.0.36 () {58 vars in 1059 bytes} [Mon Oct 17 06:56:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:18.183967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb28e12-4ca3-49e5-b654-75f4fffcc105 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:18.185594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0cb28e12-4ca3-49e5-b654-75f4fffcc105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:18.289771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb28e12-4ca3-49e5-b654-75f4fffcc105 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:18.318220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:18.363410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f9884-4bdb-4d34-902c-0a8280236b06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:18.364010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f9884-4bdb-4d34-902c-0a8280236b06 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:18.364289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f9884-4bdb-4d34-902c-0a8280236b06 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:18.364565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f9884-4bdb-4d34-902c-0a8280236b06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:18.366140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-124f9884-4bdb-4d34-902c-0a8280236b06 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:18.478405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f9884-4bdb-4d34-902c-0a8280236b06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:18.479874 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f9884-4bdb-4d34-902c-0a8280236b06 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:18.497820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 534/1073] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:19.232505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da231-1e7f-4327-be91-4148cdf2fcd4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:19.233006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da231-1e7f-4327-be91-4148cdf2fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:19.233114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da231-1e7f-4327-be91-4148cdf2fcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:19.299983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6a1da231-1e7f-4327-be91-4148cdf2fcd4 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:19.301090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6a1da231-1e7f-4327-be91-4148cdf2fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:19.391852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 540/1074] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 06:56:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:19.431936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6836-669d-4438-bdc7-a88f00a0eae4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:19.432369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6836-669d-4438-bdc7-a88f00a0eae4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:19.432470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6836-669d-4438-bdc7-a88f00a0eae4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:19.433890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 535/1075] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:19.438877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:19.439014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b38a5-304c-48b1-8605-2b29466ce03e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:19.439286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b38a5-304c-48b1-8605-2b29466ce03e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:19.439504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:19.439631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b38a5-304c-48b1-8605-2b29466ce03e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:19.439780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:19.440891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 541/1076] 10.222.0.36 () {54 vars in 943 bytes} [Mon Oct 17 06:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:19.449067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:19.449355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:19.449632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:19.502154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:19.520279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:19.521388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:19.534816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:19.557757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] MFA Rules not processed for user `5f4baa1481254765b8af7a253806f28f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:19.559021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:19.659247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 536/1077] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:56:19.683450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 542/1078] 10.222.0.36 () {58 vars in 1040 bytes} [Mon Oct 17 06:56:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:56:19.693344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:19.694534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:19.796627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4747aa-6ea4-4102-812a-4e81aac3aebe None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:19.835445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:19.900432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2892186-bd37-4e3c-bb24-46aa3edeebb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:19.901033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2892186-bd37-4e3c-bb24-46aa3edeebb0 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:19.901302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2892186-bd37-4e3c-bb24-46aa3edeebb0 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:19.901570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2892186-bd37-4e3c-bb24-46aa3edeebb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:19.903153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d2892186-bd37-4e3c-bb24-46aa3edeebb0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:19.940758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:19.941939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:19.987401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2892186-bd37-4e3c-bb24-46aa3edeebb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:19.988986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2892186-bd37-4e3c-bb24-46aa3edeebb0 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:19.997481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 537/1079] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 06:56:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 06:56:20.042071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:20.046130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc68204-2caf-46e8-8fdc-a6baf81a4328 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:20.195339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0e90f-d97f-4bb7-96ee-cff26c4e4b95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:20.196117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0e90f-d97f-4bb7-96ee-cff26c4e4b95 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:20.196489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0e90f-d97f-4bb7-96ee-cff26c4e4b95 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:20.196820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0e90f-d97f-4bb7-96ee-cff26c4e4b95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:20.198461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-73b0e90f-d97f-4bb7-96ee-cff26c4e4b95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:20.299136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0e90f-d97f-4bb7-96ee-cff26c4e4b95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:20.300753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0e90f-d97f-4bb7-96ee-cff26c4e4b95 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:20.320677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 543/1080] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 06:56:22.521628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f0a5-c27b-4a28-a13f-a0250e66702d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:22.522328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f0a5-c27b-4a28-a13f-a0250e66702d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:22.522328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f0a5-c27b-4a28-a13f-a0250e66702d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:22.523609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 538/1081] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:22.528998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c7a8-7231-4029-b98d-11a7cfc659e0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:22.529478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c7a8-7231-4029-b98d-11a7cfc659e0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:22.529573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c7a8-7231-4029-b98d-11a7cfc659e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:22.534023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:22.606887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a5f9c7a8-7231-4029-b98d-11a7cfc659e0 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:22.608672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a5f9c7a8-7231-4029-b98d-11a7cfc659e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:22.712413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 544/1082] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:56:23.000069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4f0a5-c27b-4a28-a13f-a0250e66702d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:23.001945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbb4f0a5-c27b-4a28-a13f-a0250e66702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:23.083534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:23.187580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4f0a5-c27b-4a28-a13f-a0250e66702d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:23.312858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2474f8-cda1-4dca-b6f7-80ae5041aa1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:23.313371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2474f8-cda1-4dca-b6f7-80ae5041aa1f service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:23.313654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2474f8-cda1-4dca-b6f7-80ae5041aa1f service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:23.313926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2474f8-cda1-4dca-b6f7-80ae5041aa1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:23.315539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7e2474f8-cda1-4dca-b6f7-80ae5041aa1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:23.356006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:23.417624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2474f8-cda1-4dca-b6f7-80ae5041aa1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:23.419289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2474f8-cda1-4dca-b6f7-80ae5041aa1f service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:23.437904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 539/1083] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:25.670711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c4f07-eddf-4c9a-84bb-645247679128 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:25.671282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c4f07-eddf-4c9a-84bb-645247679128 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:25.671282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c4f07-eddf-4c9a-84bb-645247679128 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:25.672707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 545/1084] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:25.678529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01baca-5dc0-4c94-b461-07856c10f618 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:25.678976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01baca-5dc0-4c94-b461-07856c10f618 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:25.679159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01baca-5dc0-4c94-b461-07856c10f618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:25.758955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7b01baca-5dc0-4c94-b461-07856c10f618 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:25.760219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7b01baca-5dc0-4c94-b461-07856c10f618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:25.871690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 540/1085] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:26.169970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c4f07-eddf-4c9a-84bb-645247679128 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:26.173079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-817c4f07-eddf-4c9a-84bb-645247679128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:26.264131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:26.336359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c4f07-eddf-4c9a-84bb-645247679128 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:26.459878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc94ba-b86f-4722-b9f6-4bc8dce0f14b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:26.460417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc94ba-b86f-4722-b9f6-4bc8dce0f14b service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:26.460697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc94ba-b86f-4722-b9f6-4bc8dce0f14b service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:26.460971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc94ba-b86f-4722-b9f6-4bc8dce0f14b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:26.462605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f5cc94ba-b86f-4722-b9f6-4bc8dce0f14b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:26.508737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:26.564522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc94ba-b86f-4722-b9f6-4bc8dce0f14b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:26.566115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc94ba-b86f-4722-b9f6-4bc8dce0f14b service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:26.585499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 546/1086] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:35.869728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:36.583016 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=26 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:42.418389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e611f2-5710-4bcc-acab-ff4b0cd92c10 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:42.418926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e611f2-5710-4bcc-acab-ff4b0cd92c10 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:42.418926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e611f2-5710-4bcc-acab-ff4b0cd92c10 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:42.420224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 541/1087] 10.222.0.36 () {54 vars in 918 bytes} [Mon Oct 17 06:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:42.425238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdcde8-3980-44a3-a20b-c4e0fcd42917 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:42.425618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdcde8-3980-44a3-a20b-c4e0fcd42917 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:42.425772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdcde8-3980-44a3-a20b-c4e0fcd42917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:42.499882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1dfdcde8-3980-44a3-a20b-c4e0fcd42917 None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:42.501065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1dfdcde8-3980-44a3-a20b-c4e0fcd42917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:42.604283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 547/1088] 10.222.0.36 () {58 vars in 1015 bytes} [Mon Oct 17 06:56:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:42.618719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ce6c0-8297-4b25-955f-bbfa47ee7a4e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:42.618999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ce6c0-8297-4b25-955f-bbfa47ee7a4e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:42.619303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ce6c0-8297-4b25-955f-bbfa47ee7a4e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:42.620610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 542/1089] 10.222.0.36 () {54 vars in 918 bytes} [Mon Oct 17 06:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:42.624999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f481-4a57-4d99-8e17-13eace7d570c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:42.625280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f481-4a57-4d99-8e17-13eace7d570c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:42.625545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f481-4a57-4d99-8e17-13eace7d570c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:42.697580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:42.700172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-aa78f481-4a57-4d99-8e17-13eace7d570c None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:42.701896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa78f481-4a57-4d99-8e17-13eace7d570c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:42.815564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 548/1090] 10.222.0.36 () {58 vars in 1015 bytes} [Mon Oct 17 06:56:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:56:45.354211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab375d2-aedb-45fe-affd-cb57e7761b9c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:45.354870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab375d2-aedb-45fe-affd-cb57e7761b9c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:45.354870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab375d2-aedb-45fe-affd-cb57e7761b9c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:45.356148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 543/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:45.361481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9944be2-c906-4b7d-a740-dcbe3e2d4f19 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:45.361926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9944be2-c906-4b7d-a740-dcbe3e2d4f19 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:45.362038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9944be2-c906-4b7d-a740-dcbe3e2d4f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:45.427999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d9944be2-c906-4b7d-a740-dcbe3e2d4f19 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:45.429359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9944be2-c906-4b7d-a740-dcbe3e2d4f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:45.483045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:45.523482 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 549/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:56:45.809579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab375d2-aedb-45fe-affd-cb57e7761b9c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:45.811130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5ab375d2-aedb-45fe-affd-cb57e7761b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:45.974326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab375d2-aedb-45fe-affd-cb57e7761b9c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:46.047072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:46.086969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de1751-99dc-45cd-9691-3bc85ea7eb86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:46.087601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de1751-99dc-45cd-9691-3bc85ea7eb86 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:46.087878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de1751-99dc-45cd-9691-3bc85ea7eb86 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:46.088148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de1751-99dc-45cd-9691-3bc85ea7eb86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:46.089723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-46de1751-99dc-45cd-9691-3bc85ea7eb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:46.178204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de1751-99dc-45cd-9691-3bc85ea7eb86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:46.180084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de1751-99dc-45cd-9691-3bc85ea7eb86 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:46.197475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 544/1093] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:47.331190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6137d-154c-4c5a-9f03-4424ca2dcb1e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:47.332272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6137d-154c-4c5a-9f03-4424ca2dcb1e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:47.332272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6137d-154c-4c5a-9f03-4424ca2dcb1e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:47.333575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 550/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:47.339807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465c7c-1a3f-427c-9cc5-a7ebbf07e281 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:47.340246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465c7c-1a3f-427c-9cc5-a7ebbf07e281 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:47.340349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465c7c-1a3f-427c-9cc5-a7ebbf07e281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:47.415304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c6465c7c-1a3f-427c-9cc5-a7ebbf07e281 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:47.416422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c6465c7c-1a3f-427c-9cc5-a7ebbf07e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:47.443151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:47.511439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 545/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:47.670561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6137d-154c-4c5a-9f03-4424ca2dcb1e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:47.671999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-27b6137d-154c-4c5a-9f03-4424ca2dcb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:47.814246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6137d-154c-4c5a-9f03-4424ca2dcb1e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:47.854262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:47.925473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae8463-b46a-4552-a902-9eee1a8e4dd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:47.925993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae8463-b46a-4552-a902-9eee1a8e4dd5 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:47.926271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae8463-b46a-4552-a902-9eee1a8e4dd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:47.926547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae8463-b46a-4552-a902-9eee1a8e4dd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:47.928154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c2ae8463-b46a-4552-a902-9eee1a8e4dd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:48.015424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae8463-b46a-4552-a902-9eee1a8e4dd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:48.016985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae8463-b46a-4552-a902-9eee1a8e4dd5 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:48.035663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 551/1096] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:49.148401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10264667-826d-48db-83a2-85c76eb8dec7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:49.148919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10264667-826d-48db-83a2-85c76eb8dec7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:49.148919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10264667-826d-48db-83a2-85c76eb8dec7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:49.150180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 546/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:49.156496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf5369-2a82-40d2-be2e-e35a7966318c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:49.156928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf5369-2a82-40d2-be2e-e35a7966318c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:49.157044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf5369-2a82-40d2-be2e-e35a7966318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:49.211245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:49.234172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8fbf5369-2a82-40d2-be2e-e35a7966318c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:49.235423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8fbf5369-2a82-40d2-be2e-e35a7966318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:49.329664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 552/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:49.478731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10264667-826d-48db-83a2-85c76eb8dec7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:49.480282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-10264667-826d-48db-83a2-85c76eb8dec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:49.624259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10264667-826d-48db-83a2-85c76eb8dec7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:49.637138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:49.736980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc89f6-3fc1-48cd-a90c-4df9c6940d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:49.737494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc89f6-3fc1-48cd-a90c-4df9c6940d33 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:49.738326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc89f6-3fc1-48cd-a90c-4df9c6940d33 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:49.738593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc89f6-3fc1-48cd-a90c-4df9c6940d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:49.740199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-82fc89f6-3fc1-48cd-a90c-4df9c6940d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:49.826832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc89f6-3fc1-48cd-a90c-4df9c6940d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:49.828450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc89f6-3fc1-48cd-a90c-4df9c6940d33 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:49.845636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 547/1099] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:50.949074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db2fb9-2b91-4200-ac60-95c0ff533cb7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:50.949626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db2fb9-2b91-4200-ac60-95c0ff533cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:50.949626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db2fb9-2b91-4200-ac60-95c0ff533cb7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:50.950997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 553/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:50.957958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e638b64-b8fc-4aa0-9639-0de8ea50ba2c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:50.958434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e638b64-b8fc-4aa0-9639-0de8ea50ba2c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:50.958536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e638b64-b8fc-4aa0-9639-0de8ea50ba2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:50.978718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:51.034367 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1e638b64-b8fc-4aa0-9639-0de8ea50ba2c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:51.036140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1e638b64-b8fc-4aa0-9639-0de8ea50ba2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:51.131179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 548/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:51.281724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db2fb9-2b91-4200-ac60-95c0ff533cb7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:51.283173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-72db2fb9-2b91-4200-ac60-95c0ff533cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:51.388662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:51.423016 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db2fb9-2b91-4200-ac60-95c0ff533cb7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:51.531641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3806c52-24f2-49de-abe1-d3cd93c016e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:51.532444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3806c52-24f2-49de-abe1-d3cd93c016e1 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:51.532603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3806c52-24f2-49de-abe1-d3cd93c016e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:51.532760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3806c52-24f2-49de-abe1-d3cd93c016e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:51.534486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3806c52-24f2-49de-abe1-d3cd93c016e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:51.611868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:51.625662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3806c52-24f2-49de-abe1-d3cd93c016e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:51.627542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3806c52-24f2-49de-abe1-d3cd93c016e1 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:51.645639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 554/1102] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:52.749612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734c876-0850-48c0-a2d5-5c19d6d8c1f5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:52.750199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734c876-0850-48c0-a2d5-5c19d6d8c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:52.750199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734c876-0850-48c0-a2d5-5c19d6d8c1f5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:52.751552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 549/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:52.758321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238cd-0006-43f6-8163-847771956144 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:52.758745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238cd-0006-43f6-8163-847771956144 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:52.758873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238cd-0006-43f6-8163-847771956144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:52.834789 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f08238cd-0006-43f6-8163-847771956144 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:52.836031 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f08238cd-0006-43f6-8163-847771956144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:52.931510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 555/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:53.085126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734c876-0850-48c0-a2d5-5c19d6d8c1f5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:53.086541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a734c876-0850-48c0-a2d5-5c19d6d8c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:53.171384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:53.232241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734c876-0850-48c0-a2d5-5c19d6d8c1f5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:53.343052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbb55c-24c4-44c8-9fc7-e02567f32971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:53.343620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb55c-24c4-44c8-9fc7-e02567f32971 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:53.343893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb55c-24c4-44c8-9fc7-e02567f32971 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:53.344158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb55c-24c4-44c8-9fc7-e02567f32971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:53.345727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-45dbb55c-24c4-44c8-9fc7-e02567f32971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:53.393240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:53.433830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbb55c-24c4-44c8-9fc7-e02567f32971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:53.435432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbb55c-24c4-44c8-9fc7-e02567f32971 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:53.452885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 550/1105] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:54.489944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032de08b-b103-4a86-8a4d-afbc293361e9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:54.490469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032de08b-b103-4a86-8a4d-afbc293361e9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:54.490469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032de08b-b103-4a86-8a4d-afbc293361e9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:54.491758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 556/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:54.497789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d25f12-aa91-4cf8-928a-ecb14bbf17e5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:54.498217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d25f12-aa91-4cf8-928a-ecb14bbf17e5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:54.498396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d25f12-aa91-4cf8-928a-ecb14bbf17e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:54.573548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-20d25f12-aa91-4cf8-928a-ecb14bbf17e5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:54.574732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-20d25f12-aa91-4cf8-928a-ecb14bbf17e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:54.666672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 551/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:54.814892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032de08b-b103-4a86-8a4d-afbc293361e9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:54.817103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-032de08b-b103-4a86-8a4d-afbc293361e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:54.866479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:54.961437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032de08b-b103-4a86-8a4d-afbc293361e9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:55.070688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72888-431c-4a20-b6b1-cc4b5e06bf8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:55.071197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72888-431c-4a20-b6b1-cc4b5e06bf8e service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:55.071504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72888-431c-4a20-b6b1-cc4b5e06bf8e service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:55.071768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72888-431c-4a20-b6b1-cc4b5e06bf8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:55.073321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dee72888-431c-4a20-b6b1-cc4b5e06bf8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:55.080036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:55.161032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee72888-431c-4a20-b6b1-cc4b5e06bf8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:55.162618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee72888-431c-4a20-b6b1-cc4b5e06bf8e service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:55.181212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 557/1108] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:56.243719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91ec1-5093-45ff-9056-cf23c6ac9872 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:56.244243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91ec1-5093-45ff-9056-cf23c6ac9872 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:56.244243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91ec1-5093-45ff-9056-cf23c6ac9872 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:56.245602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 552/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:56.251649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4098a-de15-4e2e-b2b4-5dc3651ef140 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:56.252055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4098a-de15-4e2e-b2b4-5dc3651ef140 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:56.252169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4098a-de15-4e2e-b2b4-5dc3651ef140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:56.327995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-34a4098a-de15-4e2e-b2b4-5dc3651ef140 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:56.329180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-34a4098a-de15-4e2e-b2b4-5dc3651ef140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:56.395859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:56.424410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 558/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:56:56.573471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb91ec1-5093-45ff-9056-cf23c6ac9872 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:56.574973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5eb91ec1-5093-45ff-9056-cf23c6ac9872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:56.592805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:56.718576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb91ec1-5093-45ff-9056-cf23c6ac9872 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:56.805043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:56.831723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e90921-8492-435d-93c1-cb986ea67531 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:56.832324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90921-8492-435d-93c1-cb986ea67531 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:56.832601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90921-8492-435d-93c1-cb986ea67531 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:56.832871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90921-8492-435d-93c1-cb986ea67531 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:56.834442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d0e90921-8492-435d-93c1-cb986ea67531 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:56.922949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e90921-8492-435d-93c1-cb986ea67531 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:56.924565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e90921-8492-435d-93c1-cb986ea67531 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:56.943238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 553/1111] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:57.986142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bc4a5-4c92-435a-ba09-37fd3726a2db None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:57.986715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bc4a5-4c92-435a-ba09-37fd3726a2db None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:57.986715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bc4a5-4c92-435a-ba09-37fd3726a2db None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:57.988742 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 559/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:57.995053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c799-8564-44b5-8175-cbb346cec873 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:57.995533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c799-8564-44b5-8175-cbb346cec873 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:57.995632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c799-8564-44b5-8175-cbb346cec873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:58.070308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7a21c799-8564-44b5-8175-cbb346cec873 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:58.071576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7a21c799-8564-44b5-8175-cbb346cec873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:58.113411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:58.167457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 554/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:56:58.317038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3bc4a5-4c92-435a-ba09-37fd3726a2db None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:56:58.318460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4a3bc4a5-4c92-435a-ba09-37fd3726a2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:58.459387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3bc4a5-4c92-435a-ba09-37fd3726a2db None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:56:58.513209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:58.572579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1178-cd79-4a03-bf95-4c33e5a38dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:56:58.573112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1178-cd79-4a03-bf95-4c33e5a38dff service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:58.573389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1178-cd79-4a03-bf95-4c33e5a38dff service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:58.573660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1178-cd79-4a03-bf95-4c33e5a38dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:58.575278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-becb1178-cd79-4a03-bf95-4c33e5a38dff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:58.663939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1178-cd79-4a03-bf95-4c33e5a38dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:56:58.665505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1178-cd79-4a03-bf95-4c33e5a38dff service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:56:58.683586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 560/1114] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:56:59.732638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289dd2c-70ee-43af-af1b-0055bd158a8f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:59.733200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289dd2c-70ee-43af-af1b-0055bd158a8f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:59.733200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289dd2c-70ee-43af-af1b-0055bd158a8f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:59.734545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 555/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 06:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:56:59.741328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3932-d9a3-4283-abac-b0a54764aa74 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:56:59.741772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3932-d9a3-4283-abac-b0a54764aa74 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:56:59.741875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3932-d9a3-4283-abac-b0a54764aa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:56:59.815704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-19ea3932-d9a3-4283-abac-b0a54764aa74 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:56:59.816909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-19ea3932-d9a3-4283-abac-b0a54764aa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:56:59.818859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:56:59.910885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 561/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 06:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:00.062544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289dd2c-70ee-43af-af1b-0055bd158a8f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:00.064035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3289dd2c-70ee-43af-af1b-0055bd158a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:00.211422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289dd2c-70ee-43af-af1b-0055bd158a8f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:00.238429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:00.322015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d1e16-6f50-43cb-aa95-dfe8b5110c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:00.322574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d1e16-6f50-43cb-aa95-dfe8b5110c8d service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:00.322848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d1e16-6f50-43cb-aa95-dfe8b5110c8d service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:00.323112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d1e16-6f50-43cb-aa95-dfe8b5110c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:00.324699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-da6d1e16-6f50-43cb-aa95-dfe8b5110c8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:00.411447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d1e16-6f50-43cb-aa95-dfe8b5110c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:00.413022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d1e16-6f50-43cb-aa95-dfe8b5110c8d service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:00.431641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 556/1117] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:02.477541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06d43-d5b6-4c61-98f7-e33c21c049ac None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:02.478063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06d43-d5b6-4c61-98f7-e33c21c049ac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:02.478063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06d43-d5b6-4c61-98f7-e33c21c049ac None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:02.479343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 562/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:02.484559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101ba3d-59b5-45bf-b586-465340b77c1f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:02.484953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101ba3d-59b5-45bf-b586-465340b77c1f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:02.485106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101ba3d-59b5-45bf-b586-465340b77c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:02.526323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:02.550946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d101ba3d-59b5-45bf-b586-465340b77c1f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:02.552167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d101ba3d-59b5-45bf-b586-465340b77c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:02.646634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 557/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:02.930423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b06d43-d5b6-4c61-98f7-e33c21c049ac None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:02.931981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f4b06d43-d5b6-4c61-98f7-e33c21c049ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:03.031791 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b06d43-d5b6-4c61-98f7-e33c21c049ac None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:03.063116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:03.103714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee304b9c-f341-4081-a308-480e6209c491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:03.104308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee304b9c-f341-4081-a308-480e6209c491 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:03.104579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee304b9c-f341-4081-a308-480e6209c491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:03.104840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee304b9c-f341-4081-a308-480e6209c491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:03.106390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ee304b9c-f341-4081-a308-480e6209c491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:03.217836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee304b9c-f341-4081-a308-480e6209c491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:03.219273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee304b9c-f341-4081-a308-480e6209c491 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:03.237512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 563/1120] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:05.529411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f433681-d9da-4cf4-a620-4f0280b1975e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:05.529937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f433681-d9da-4cf4-a620-4f0280b1975e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:05.529937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f433681-d9da-4cf4-a620-4f0280b1975e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:05.531202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 558/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:05.537069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64310b6-1903-4fba-b54a-ea6c9e19b638 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:05.537460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64310b6-1903-4fba-b54a-ea6c9e19b638 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:05.537610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64310b6-1903-4fba-b54a-ea6c9e19b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:05.601943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b64310b6-1903-4fba-b54a-ea6c9e19b638 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:05.603136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b64310b6-1903-4fba-b54a-ea6c9e19b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:05.616094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:05.698350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 564/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:05.987231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f433681-d9da-4cf4-a620-4f0280b1975e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:05.988790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4f433681-d9da-4cf4-a620-4f0280b1975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:06.090892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f433681-d9da-4cf4-a620-4f0280b1975e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:06.104526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:06.166001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced08c89-f78e-4385-9eb2-84a4ea3c0c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:06.166605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced08c89-f78e-4385-9eb2-84a4ea3c0c9c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:06.166881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced08c89-f78e-4385-9eb2-84a4ea3c0c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:06.167152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced08c89-f78e-4385-9eb2-84a4ea3c0c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:06.168754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ced08c89-f78e-4385-9eb2-84a4ea3c0c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:06.275255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced08c89-f78e-4385-9eb2-84a4ea3c0c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:06.276610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced08c89-f78e-4385-9eb2-84a4ea3c0c9c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:06.294286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 559/1123] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:08.740091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5239-ee2a-4691-bef1-b95a26dae804 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:08.740652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5239-ee2a-4691-bef1-b95a26dae804 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:08.740652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5239-ee2a-4691-bef1-b95a26dae804 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:08.741985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 565/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:08.747109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7a5b6-00fa-4fce-86fa-525658fefe08 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:08.747571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7a5b6-00fa-4fce-86fa-525658fefe08 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:08.747702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7a5b6-00fa-4fce-86fa-525658fefe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:08.796083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:08.814482 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-99c7a5b6-00fa-4fce-86fa-525658fefe08 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:08.816262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-99c7a5b6-00fa-4fce-86fa-525658fefe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:08.910036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 560/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:09.197071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a5239-ee2a-4691-bef1-b95a26dae804 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:09.198596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1f2a5239-ee2a-4691-bef1-b95a26dae804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:09.300001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a5239-ee2a-4691-bef1-b95a26dae804 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:09.339993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:09.372975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5603e67-d65d-471c-8918-b54c0a531396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:09.373577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603e67-d65d-471c-8918-b54c0a531396 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:09.373855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603e67-d65d-471c-8918-b54c0a531396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:09.374125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603e67-d65d-471c-8918-b54c0a531396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:09.375735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d5603e67-d65d-471c-8918-b54c0a531396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:09.482150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5603e67-d65d-471c-8918-b54c0a531396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:09.483520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5603e67-d65d-471c-8918-b54c0a531396 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:09.501350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 566/1126] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:11.761186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d754f-c7e9-4754-b11b-25fe59d099d8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:11.761760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d754f-c7e9-4754-b11b-25fe59d099d8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:11.761760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d754f-c7e9-4754-b11b-25fe59d099d8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:11.763064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 561/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:11.768904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428882-1125-4de8-94d2-fe3f35918dff None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:11.769301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428882-1125-4de8-94d2-fe3f35918dff None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:11.769441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428882-1125-4de8-94d2-fe3f35918dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:11.834318 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-01428882-1125-4de8-94d2-fe3f35918dff None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:11.835524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-01428882-1125-4de8-94d2-fe3f35918dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:11.855390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:11.929805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 567/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:57:12.216453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d754f-c7e9-4754-b11b-25fe59d099d8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:12.218046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6e8d754f-c7e9-4754-b11b-25fe59d099d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:12.319274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d754f-c7e9-4754-b11b-25fe59d099d8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:12.342075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:12.394681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e611-cc5d-40f9-9b81-5a8a61a449dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:12.395299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e611-cc5d-40f9-9b81-5a8a61a449dc service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:12.395573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e611-cc5d-40f9-9b81-5a8a61a449dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:12.395839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e611-cc5d-40f9-9b81-5a8a61a449dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:12.397389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-34d1e611-cc5d-40f9-9b81-5a8a61a449dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:12.502074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e611-cc5d-40f9-9b81-5a8a61a449dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:12.503961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e611-cc5d-40f9-9b81-5a8a61a449dc service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:12.521163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 562/1129] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:14.722256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35bf23-e822-47a3-819c-357119daadb8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:14.722841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35bf23-e822-47a3-819c-357119daadb8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:14.722841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35bf23-e822-47a3-819c-357119daadb8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:14.724163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 568/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:14.729630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995b66-efc1-4480-8e7e-e672634bb159 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:14.730047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995b66-efc1-4480-8e7e-e672634bb159 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:14.730155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995b66-efc1-4480-8e7e-e672634bb159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:14.792550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:14.795400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d6995b66-efc1-4480-8e7e-e672634bb159 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:14.797124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d6995b66-efc1-4480-8e7e-e672634bb159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:14.891952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 563/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:15.179226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35bf23-e822-47a3-819c-357119daadb8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:15.181023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bf35bf23-e822-47a3-819c-357119daadb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:15.282200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35bf23-e822-47a3-819c-357119daadb8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:15.330436 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:15.354558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35d9df-65a0-49a6-b8b1-447e556605af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:15.355149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d9df-65a0-49a6-b8b1-447e556605af service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:15.355451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d9df-65a0-49a6-b8b1-447e556605af service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:15.355718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d9df-65a0-49a6-b8b1-447e556605af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:15.357293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6b35d9df-65a0-49a6-b8b1-447e556605af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:15.462493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35d9df-65a0-49a6-b8b1-447e556605af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:15.463897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35d9df-65a0-49a6-b8b1-447e556605af service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:15.481752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 569/1132] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:17.673384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0b880-4b77-4535-87ec-1b0af7590021 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:17.673888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0b880-4b77-4535-87ec-1b0af7590021 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:17.674000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0b880-4b77-4535-87ec-1b0af7590021 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:17.675898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 564/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:17.681443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df40d-5c7b-49c8-8acd-d9e54b147f82 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:17.681901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df40d-5c7b-49c8-8acd-d9e54b147f82 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:17.681998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df40d-5c7b-49c8-8acd-d9e54b147f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:17.747043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-114df40d-5c7b-49c8-8acd-d9e54b147f82 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:17.748421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-114df40d-5c7b-49c8-8acd-d9e54b147f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:17.776491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:17.842441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 570/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:18.124367 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0b880-4b77-4535-87ec-1b0af7590021 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:18.125888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-34e0b880-4b77-4535-87ec-1b0af7590021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:18.229387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0b880-4b77-4535-87ec-1b0af7590021 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:18.257317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:18.301243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a094f-8566-403d-99e0-85a837210d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:18.301838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a094f-8566-403d-99e0-85a837210d23 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:18.302111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a094f-8566-403d-99e0-85a837210d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:18.302377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a094f-8566-403d-99e0-85a837210d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:18.303979 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-815a094f-8566-403d-99e0-85a837210d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:18.411297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a094f-8566-403d-99e0-85a837210d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:18.412726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a094f-8566-403d-99e0-85a837210d23 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:18.430333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 565/1135] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:20.616369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920f8ea-d3ed-4224-a60d-82c41f6863a7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:20.616944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920f8ea-d3ed-4224-a60d-82c41f6863a7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:20.616944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920f8ea-d3ed-4224-a60d-82c41f6863a7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:20.618260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 571/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:20.623610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fab3f-2356-47f9-a853-26115389b3ae None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:20.624032 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fab3f-2356-47f9-a853-26115389b3ae None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:20.624140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fab3f-2356-47f9-a853-26115389b3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:20.689371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5b6fab3f-2356-47f9-a853-26115389b3ae None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:20.690562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b6fab3f-2356-47f9-a853-26115389b3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:20.698761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:20.785263 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 566/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:21.066527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920f8ea-d3ed-4224-a60d-82c41f6863a7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:21.068058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7920f8ea-d3ed-4224-a60d-82c41f6863a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:21.171657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920f8ea-d3ed-4224-a60d-82c41f6863a7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:21.227268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:21.245628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b577d-0135-4a8f-b796-1857d501d0c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:21.246225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b577d-0135-4a8f-b796-1857d501d0c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:21.246498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b577d-0135-4a8f-b796-1857d501d0c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:21.246764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b577d-0135-4a8f-b796-1857d501d0c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:21.248373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a50b577d-0135-4a8f-b796-1857d501d0c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:21.353053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b577d-0135-4a8f-b796-1857d501d0c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:21.354410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b577d-0135-4a8f-b796-1857d501d0c2 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:21.372278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 572/1138] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:23.559940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50704-1962-4d3f-ab46-192f009f4fa5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:23.560526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50704-1962-4d3f-ab46-192f009f4fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:23.560526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50704-1962-4d3f-ab46-192f009f4fa5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:23.561813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 567/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:23.567116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552135c-dae9-44c1-b405-cd7c14e960c5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:23.567585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552135c-dae9-44c1-b405-cd7c14e960c5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:23.567693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552135c-dae9-44c1-b405-cd7c14e960c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:23.632514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8552135c-dae9-44c1-b405-cd7c14e960c5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:23.633700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8552135c-dae9-44c1-b405-cd7c14e960c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:23.672005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:23.728301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 573/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:24.010206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c50704-1962-4d3f-ab46-192f009f4fa5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:24.011769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d5c50704-1962-4d3f-ab46-192f009f4fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:24.114743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c50704-1962-4d3f-ab46-192f009f4fa5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:24.151687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:24.188300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4dbf2-6d0c-4a1d-ae55-fbeeed094fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:24.188896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4dbf2-6d0c-4a1d-ae55-fbeeed094fcb service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:24.189166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4dbf2-6d0c-4a1d-ae55-fbeeed094fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:24.189428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4dbf2-6d0c-4a1d-ae55-fbeeed094fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:24.190981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e0b4dbf2-6d0c-4a1d-ae55-fbeeed094fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:24.296570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4dbf2-6d0c-4a1d-ae55-fbeeed094fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:24.297927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4dbf2-6d0c-4a1d-ae55-fbeeed094fcb service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:24.315823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 568/1141] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:26.448884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0df4f2-ce89-4bc4-a9f0-9e18b4a755a1 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:26.449429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0df4f2-ce89-4bc4-a9f0-9e18b4a755a1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:26.449429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0df4f2-ce89-4bc4-a9f0-9e18b4a755a1 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:26.450719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 574/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:57:26.456029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95c4e-e03a-47a7-acc2-cc38cb857334 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:26.456429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95c4e-e03a-47a7-acc2-cc38cb857334 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:26.456562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95c4e-e03a-47a7-acc2-cc38cb857334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:26.522569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e1f95c4e-e03a-47a7-acc2-cc38cb857334 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:26.523786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e1f95c4e-e03a-47a7-acc2-cc38cb857334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:26.541284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:26.618473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 569/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:26.902395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0df4f2-ce89-4bc4-a9f0-9e18b4a755a1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:26.903955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4b0df4f2-ce89-4bc4-a9f0-9e18b4a755a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:27.005514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0df4f2-ce89-4bc4-a9f0-9e18b4a755a1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:27.071285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:27.078325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46c1b2-66e3-4973-afd6-4edd327933e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:27.078915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46c1b2-66e3-4973-afd6-4edd327933e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:27.079193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46c1b2-66e3-4973-afd6-4edd327933e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:27.079496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46c1b2-66e3-4973-afd6-4edd327933e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:27.081076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ff46c1b2-66e3-4973-afd6-4edd327933e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:27.190094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46c1b2-66e3-4973-afd6-4edd327933e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:27.191562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46c1b2-66e3-4973-afd6-4edd327933e3 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:27.208970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 575/1144] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:29.349790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d452c89-8204-465a-b175-50b901ec78e8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:29.350322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d452c89-8204-465a-b175-50b901ec78e8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:29.350460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d452c89-8204-465a-b175-50b901ec78e8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:29.351749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 570/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 06:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:29.357151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd1aa-93f1-4eba-89df-7acf15763fdc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:29.357548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd1aa-93f1-4eba-89df-7acf15763fdc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:29.357698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd1aa-93f1-4eba-89df-7acf15763fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:29.421653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-873dd1aa-93f1-4eba-89df-7acf15763fdc None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:29.422749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-873dd1aa-93f1-4eba-89df-7acf15763fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:29.470803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:29.517431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 576/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 06:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:29.797749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d452c89-8204-465a-b175-50b901ec78e8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:29.799338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0d452c89-8204-465a-b175-50b901ec78e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:29.903088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d452c89-8204-465a-b175-50b901ec78e8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:29.950808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:29.975959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2bc67-19d2-4f09-938a-ae2b8495b11a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:29.976551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2bc67-19d2-4f09-938a-ae2b8495b11a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:29.976829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2bc67-19d2-4f09-938a-ae2b8495b11a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:29.977101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2bc67-19d2-4f09-938a-ae2b8495b11a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:29.978668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c6a2bc67-19d2-4f09-938a-ae2b8495b11a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:30.086458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2bc67-19d2-4f09-938a-ae2b8495b11a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:30.087847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2bc67-19d2-4f09-938a-ae2b8495b11a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:30.105449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 571/1147] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:39.516044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=22 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:40.104090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=66 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:49.910580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9a2d0-060c-4493-9f69-d4a53a607174 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:49.911184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9a2d0-060c-4493-9f69-d4a53a607174 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:49.911184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9a2d0-060c-4493-9f69-d4a53a607174 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:49.912485 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 577/1148] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:49.917754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05205a5-3f0e-4a7f-bdbe-4f9f44463438 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:49.918188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05205a5-3f0e-4a7f-bdbe-4f9f44463438 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:49.918295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05205a5-3f0e-4a7f-bdbe-4f9f44463438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:49.983873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f05205a5-3f0e-4a7f-bdbe-4f9f44463438 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:49.985057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f05205a5-3f0e-4a7f-bdbe-4f9f44463438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:50.080182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 572/1149] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:57:50.373716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bbe16-6969-483d-a1ac-81e3dcad58ef None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:50.373998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bbe16-6969-483d-a1ac-81e3dcad58ef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:50.374268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bbe16-6969-483d-a1ac-81e3dcad58ef None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:50.375618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 578/1150] 10.222.0.36 () {54 vars in 962 bytes} [Mon Oct 17 06:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:50.381039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b62ab-eb99-4142-b20c-e68a9c36aca6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:50.381321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b62ab-eb99-4142-b20c-e68a9c36aca6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:50.381594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b62ab-eb99-4142-b20c-e68a9c36aca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:50.456155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7f8b62ab-eb99-4142-b20c-e68a9c36aca6 None None] MFA Rules not processed for user `a0a3956bb98749e6bb19ee7f3465eac9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:50.457256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7f8b62ab-eb99-4142-b20c-e68a9c36aca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:50.477387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:50.534109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 573/1151] 10.222.0.36 () {58 vars in 1059 bytes} [Mon Oct 17 06:57:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:50.542390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bbe16-6969-483d-a1ac-81e3dcad58ef None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:50.543952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-446bbe16-6969-483d-a1ac-81e3dcad58ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:50.644782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bbe16-6969-483d-a1ac-81e3dcad58ef None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:50.716804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc41daf-08e9-489f-9646-1429dc2f9ea6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:50.717321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc41daf-08e9-489f-9646-1429dc2f9ea6 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:50.717597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc41daf-08e9-489f-9646-1429dc2f9ea6 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:50.717865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc41daf-08e9-489f-9646-1429dc2f9ea6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:50.719472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3cc41daf-08e9-489f-9646-1429dc2f9ea6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:50.775640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:50.830380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc41daf-08e9-489f-9646-1429dc2f9ea6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:50.831919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc41daf-08e9-489f-9646-1429dc2f9ea6 service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:50.850667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 579/1152] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 06:57:53.074643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0a69-ef54-446f-842f-4214a939bf86 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:53.075186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0a69-ef54-446f-842f-4214a939bf86 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:53.075186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0a69-ef54-446f-842f-4214a939bf86 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:53.076466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 574/1153] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:53.081708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b91f04-6493-4de2-9651-99898e51e5fe None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:53.082117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b91f04-6493-4de2-9651-99898e51e5fe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:53.082285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b91f04-6493-4de2-9651-99898e51e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:53.148819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d0b91f04-6493-4de2-9651-99898e51e5fe None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:53.150037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d0b91f04-6493-4de2-9651-99898e51e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:53.249027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 580/1154] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:57:53.526451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb0a69-ef54-446f-842f-4214a939bf86 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:53.528045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4dfb0a69-ef54-446f-842f-4214a939bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:53.650066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb0a69-ef54-446f-842f-4214a939bf86 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:53.700328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:53.744481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d16eb-8f2c-4d92-b3e4-3e5e33e10bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:53.745104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d16eb-8f2c-4d92-b3e4-3e5e33e10bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:53.745381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d16eb-8f2c-4d92-b3e4-3e5e33e10bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:53.745655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d16eb-8f2c-4d92-b3e4-3e5e33e10bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:53.746278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d16eb-8f2c-4d92-b3e4-3e5e33e10bf7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:53.747436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-ec2d16eb-8f2c-4d92-b3e4-3e5e33e10bf7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:57:53.747436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 06:57:53.747436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 06:57:53.748585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d16eb-8f2c-4d92-b3e4-3e5e33e10bf7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:53.767189 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 575/1155] 10.222.0.36 () {56 vars in 1108 bytes} [Mon Oct 17 06:57:53 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:57:55.912971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e2fd0-c3f4-48d5-873d-cc52dc3e112f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:55.913510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e2fd0-c3f4-48d5-873d-cc52dc3e112f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:55.913510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e2fd0-c3f4-48d5-873d-cc52dc3e112f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:55.914911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 581/1156] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:55.920079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648054-c2eb-47a6-90f4-f1c46e2ca492 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:55.920497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648054-c2eb-47a6-90f4-f1c46e2ca492 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:55.920693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648054-c2eb-47a6-90f4-f1c46e2ca492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:55.986956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-21648054-c2eb-47a6-90f4-f1c46e2ca492 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:55.988200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-21648054-c2eb-47a6-90f4-f1c46e2ca492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:56.082825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 576/1157] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:57:56.374862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e2fd0-c3f4-48d5-873d-cc52dc3e112f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:56.376415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-de8e2fd0-c3f4-48d5-873d-cc52dc3e112f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:56.386588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:56.520401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e2fd0-c3f4-48d5-873d-cc52dc3e112f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:56.598205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:56.630571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deda81b-08c9-4ac8-90d6-925c5ff2e292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:56.631102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deda81b-08c9-4ac8-90d6-925c5ff2e292 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:56.631401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deda81b-08c9-4ac8-90d6-925c5ff2e292 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:56.631669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deda81b-08c9-4ac8-90d6-925c5ff2e292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:56.633228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8deda81b-08c9-4ac8-90d6-925c5ff2e292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:56.718892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deda81b-08c9-4ac8-90d6-925c5ff2e292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:56.720490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deda81b-08c9-4ac8-90d6-925c5ff2e292 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:56.738178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 582/1158] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:57:59.037200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16875b0-c67f-4c89-83e6-dbab1ffcb9d5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:59.037768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16875b0-c67f-4c89-83e6-dbab1ffcb9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:59.037768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16875b0-c67f-4c89-83e6-dbab1ffcb9d5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:59.039081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 577/1159] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:57:59.044416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228187e5-d93e-494f-b4be-48fc010251ea None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:59.044819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228187e5-d93e-494f-b4be-48fc010251ea None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:59.044913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228187e5-d93e-494f-b4be-48fc010251ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:59.110895 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-228187e5-d93e-494f-b4be-48fc010251ea None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:57:59.112123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-228187e5-d93e-494f-b4be-48fc010251ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:59.158212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:59.210259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 583/1160] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:57:59.494378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16875b0-c67f-4c89-83e6-dbab1ffcb9d5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:57:59.495929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c16875b0-c67f-4c89-83e6-dbab1ffcb9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:59.519801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:59.599558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16875b0-c67f-4c89-83e6-dbab1ffcb9d5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:57:59.670818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782cb96-4bcd-4d60-bd1d-a797329c61fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:57:59.671456 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782cb96-4bcd-4d60-bd1d-a797329c61fa service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:57:59.671732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782cb96-4bcd-4d60-bd1d-a797329c61fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:57:59.672000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782cb96-4bcd-4d60-bd1d-a797329c61fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:57:59.673571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a782cb96-4bcd-4d60-bd1d-a797329c61fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:57:59.746739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:57:59.782017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782cb96-4bcd-4d60-bd1d-a797329c61fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:57:59.783483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782cb96-4bcd-4d60-bd1d-a797329c61fa service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:57:59.801002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 578/1161] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:58:01.996552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826e535-ef20-448e-89e5-3412780522eb None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:01.997126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826e535-ef20-448e-89e5-3412780522eb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:01.997126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826e535-ef20-448e-89e5-3412780522eb None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:01.998442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 584/1162] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:58:02.003758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a3bd9-e1f2-4c1e-929b-cd21ca52d4da None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:02.004182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a3bd9-e1f2-4c1e-929b-cd21ca52d4da None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:02.004308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a3bd9-e1f2-4c1e-929b-cd21ca52d4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:02.070426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2c7a3bd9-e1f2-4c1e-929b-cd21ca52d4da None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:58:02.071677 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c7a3bd9-e1f2-4c1e-929b-cd21ca52d4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:02.168014 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 579/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:58:02.455063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826e535-ef20-448e-89e5-3412780522eb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:58:02.456639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2826e535-ef20-448e-89e5-3412780522eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:02.557802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826e535-ef20-448e-89e5-3412780522eb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:58:02.628568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:58:02.630020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8f5f2-ece6-47eb-8e84-5c378e941b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:58:02.631271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f5f2-ece6-47eb-8e84-5c378e941b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:02.631818 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f5f2-ece6-47eb-8e84-5c378e941b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:02.632110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f5f2-ece6-47eb-8e84-5c378e941b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:02.634081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-99d8f5f2-ece6-47eb-8e84-5c378e941b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:02.739158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8f5f2-ece6-47eb-8e84-5c378e941b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:58:02.740525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8f5f2-ece6-47eb-8e84-5c378e941b09 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:58:02.758239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 585/1164] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:58:05.043074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b30a22-7272-46c8-82c0-17c113dcf7eb None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:05.043664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b30a22-7272-46c8-82c0-17c113dcf7eb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:05.043664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b30a22-7272-46c8-82c0-17c113dcf7eb None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:05.044975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 580/1165] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:58:05.050218 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081b8e-fa81-4612-b3f1-187f32863b0e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:05.050645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081b8e-fa81-4612-b3f1-187f32863b0e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:05.050753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081b8e-fa81-4612-b3f1-187f32863b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:05.116350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad081b8e-fa81-4612-b3f1-187f32863b0e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:58:05.117539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad081b8e-fa81-4612-b3f1-187f32863b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:05.169398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:58:05.214341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 586/1166] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:58:05.504485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b30a22-7272-46c8-82c0-17c113dcf7eb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:58:05.506001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-41b30a22-7272-46c8-82c0-17c113dcf7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:05.541510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:58:05.607976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b30a22-7272-46c8-82c0-17c113dcf7eb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:58:05.679607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e68b4-6a49-4071-8305-ea16538ac06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:58:05.680203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e68b4-6a49-4071-8305-ea16538ac06d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:05.680478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e68b4-6a49-4071-8305-ea16538ac06d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:05.680747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e68b4-6a49-4071-8305-ea16538ac06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:05.682305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-913e68b4-6a49-4071-8305-ea16538ac06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:05.761532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:58:05.790945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e68b4-6a49-4071-8305-ea16538ac06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:58:05.792413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e68b4-6a49-4071-8305-ea16538ac06d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:58:05.811111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 581/1167] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:58:08.111230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef796d-6660-406d-989a-6af9752f7f0f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:08.111820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef796d-6660-406d-989a-6af9752f7f0f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:08.111921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef796d-6660-406d-989a-6af9752f7f0f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:08.113286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 587/1168] 10.222.0.36 () {54 vars in 927 bytes} [Mon Oct 17 06:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:58:08.118906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66575fd-d799-4305-bba4-80029589436e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:08.119360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66575fd-d799-4305-bba4-80029589436e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:08.119492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66575fd-d799-4305-bba4-80029589436e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:08.186037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b66575fd-d799-4305-bba4-80029589436e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:58:08.187255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b66575fd-d799-4305-bba4-80029589436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:08.280355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 582/1169] 10.222.0.36 () {58 vars in 1024 bytes} [Mon Oct 17 06:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:58:08.567077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef796d-6660-406d-989a-6af9752f7f0f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:58:08.569135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14ef796d-6660-406d-989a-6af9752f7f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:08.668564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef796d-6660-406d-989a-6af9752f7f0f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:58:08.739143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e8934-0a3d-4726-81f6-6e13ea729e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:58:08.739760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8934-0a3d-4726-81f6-6e13ea729e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:58:08.740032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8934-0a3d-4726-81f6-6e13ea729e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:58:08.740296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8934-0a3d-4726-81f6-6e13ea729e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:58:08.741846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba3e8934-0a3d-4726-81f6-6e13ea729e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:58:08.750846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:58:08.850052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e8934-0a3d-4726-81f6-6e13ea729e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:58:08.851438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e8934-0a3d-4726-81f6-6e13ea729e33 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:58:08.869997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 588/1170] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:58:18.278808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=82 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:58:18.867522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=50 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:28.358605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:28.359233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:28.359233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:28.434170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:59:28.435308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:28.528859 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 583/1171] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 06:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:59:28.536587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368efe9-3434-4f7f-9ac9-86fbabe2994f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:28.537076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368efe9-3434-4f7f-9ac9-86fbabe2994f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:28.537184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368efe9-3434-4f7f-9ac9-86fbabe2994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:28.611172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0368efe9-3434-4f7f-9ac9-86fbabe2994f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:59:28.612390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0368efe9-3434-4f7f-9ac9-86fbabe2994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:28.706792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 589/1172] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 06:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:59:28.711313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:28.712501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:28.723706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc25e7-c839-446b-9a3e-40e57d9bbac0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:28.784798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:28.818502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:28.819100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:28.819405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:28.819676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:28.820305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:59:28.820603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:28.821610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:59:28.821610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 06:59:28.821610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 06:59:28.822528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:28.834127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 584/1173] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 06:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:59:28.843610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368efe9-3434-4f7f-9ac9-86fbabe2994f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:28.852546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368efe9-3434-4f7f-9ac9-86fbabe2994f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:28.912904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:28.946400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:28.947017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:28.947316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:28.947582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:28.948210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:28.949299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:59:28.949299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 06:59:28.949299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 06:59:28.950480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:29.004291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 590/1174] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 06:59:28 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:59:29.008340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:29.010395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:29.042826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c19aa-7d63-46ed-ad3d-9ae0bacf2d92 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:29.135416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:29.136009 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:29.136284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:29.136551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:29.137173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:29.138175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:59:29.138175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 06:59:29.138175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 06:59:29.139399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:29.183272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 585/1175] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 06:59:29 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:59:29.187338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:29.188870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:29.251258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8da858-065f-42a8-b38d-42d40a389bcd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:29.332958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:29.343554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:29.344152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:29.344426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:29.344689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:29.345312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:29.346886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:29.359001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 591/1176] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 06:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:59:29.363286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:29.364782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:29.376490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368faf4a-8c4a-4b65-afd0-92c824500041 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:29.401323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:29.470489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f2c54-59fe-4443-97a1-68802cae48a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:29.471087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f2c54-59fe-4443-97a1-68802cae48a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:29.471391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f2c54-59fe-4443-97a1-68802cae48a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:29.471666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f2c54-59fe-4443-97a1-68802cae48a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:29.472296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f2c54-59fe-4443-97a1-68802cae48a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:29.473895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f2c54-59fe-4443-97a1-68802cae48a2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:29.486273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 586/1177] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 06:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:59:29.490430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:29.491985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:29.524802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e32932-f689-4a09-b702-192ed4cabf82 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:29.616836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf9e7e-f8cb-4551-85c3-a27f042c1705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:29.617449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf9e7e-f8cb-4551-85c3-a27f042c1705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:29.617678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf9e7e-f8cb-4551-85c3-a27f042c1705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:29.617949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf9e7e-f8cb-4551-85c3-a27f042c1705 admin admin] PATH_INFO: `/v3/projects/5e563923f9ee48cfa32fda247483bd80/users/bd83ac42b9014aacbc1a4eaab6b531b5/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:29.640217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf9e7e-f8cb-4551-85c3-a27f042c1705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e563923f9ee48cfa32fda247483bd80, user_id=bd83ac42b9014aacbc1a4eaab6b531b5, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:29.642871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf9e7e-f8cb-4551-85c3-a27f042c1705 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:29.666979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 592/1178] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 06:59:29 2022] PUT /identity/v3/projects/5e563923f9ee48cfa32fda247483bd80/users/bd83ac42b9014aacbc1a4eaab6b531b5/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:59:29.677889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c2288-bf61-4785-bec8-1934588a8db4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:29.678191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c2288-bf61-4785-bec8-1934588a8db4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:29.678513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c2288-bf61-4785-bec8-1934588a8db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:29.734995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-010c2288-bf61-4785-bec8-1934588a8db4 None None] MFA Rules not processed for user `bd83ac42b9014aacbc1a4eaab6b531b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:59:29.736219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-010c2288-bf61-4785-bec8-1934588a8db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:29.812523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 587/1179] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 06:59:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:59:29.826108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e1b52-3ccb-46f6-94d8-5cf2b9e8a0d0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:29.826392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e1b52-3ccb-46f6-94d8-5cf2b9e8a0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:29.826666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e1b52-3ccb-46f6-94d8-5cf2b9e8a0d0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:29.828020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 593/1180] 10.222.0.36 () {54 vars in 963 bytes} [Mon Oct 17 06:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:59:29.832801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71f460-225f-4daa-9dbf-0f1c73ee7089 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:29.833079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71f460-225f-4daa-9dbf-0f1c73ee7089 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:29.833349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71f460-225f-4daa-9dbf-0f1c73ee7089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:29.865133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:29.908665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9d71f460-225f-4daa-9dbf-0f1c73ee7089 None None] MFA Rules not processed for user `92c66f0a106f4c8e99ac7075d76767d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:59:29.909853 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9d71f460-225f-4daa-9dbf-0f1c73ee7089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:29.984409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 588/1181] 10.222.0.36 () {58 vars in 1060 bytes} [Mon Oct 17 06:59:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:59:29.992141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e1b52-3ccb-46f6-94d8-5cf2b9e8a0d0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:29.993635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a62e1b52-3ccb-46f6-94d8-5cf2b9e8a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:30.043463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:30.096813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e1b52-3ccb-46f6-94d8-5cf2b9e8a0d0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:30.171989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:30.172515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:30.172788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:30.173052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:30.174676 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:30.271926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:30.273334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:30.277279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:30.291683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 594/1182] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:59:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:59:30.336663 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ca6c-1ded-41d9-8ed8-4c83f3fa4eec None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:30.336942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ca6c-1ded-41d9-8ed8-4c83f3fa4eec None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:30.337214 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ca6c-1ded-41d9-8ed8-4c83f3fa4eec None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:30.339071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 589/1183] 10.222.0.36 () {52 vars in 830 bytes} [Mon Oct 17 06:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:59:30.342667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:30.343900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:30.439952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0d74-d8f1-473d-b0f9-0ccd7ea85e8d service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:30.531142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:30.531775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:30.532049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:30.532319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] PATH_INFO: `/v3/users/bd83ac42b9014aacbc1a4eaab6b531b5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:30.544765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd83ac42b9014aacbc1a4eaab6b531b5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:30.545804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:59:30.545804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 06:59:30.545804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 06:59:30.547014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:30.557069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:30.611406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd83ac42b9014aacbc1a4eaab6b531b5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:30.611928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd83ac42b9014aacbc1a4eaab6b531b5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:30.626473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd83ac42b9014aacbc1a4eaab6b531b5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:30.627047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd83ac42b9014aacbc1a4eaab6b531b5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:30.649939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd83ac42b9014aacbc1a4eaab6b531b5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:30.664653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 595/1184] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 06:59:30 2022] DELETE /identity/v3/users/bd83ac42b9014aacbc1a4eaab6b531b5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:59:30.669162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:30.669669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:30.669801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:30.726608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:59:30.727753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:30.800420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:30.814013 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 590/1185] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 06:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:59:30.824419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:30.825755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:30.923740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af077ae7-2292-458f-9eac-c9fbd4fe7dd1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:30.976133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:30.997219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d76361-4808-4bfc-8aa7-0cd3fbd43b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:30.997764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d76361-4808-4bfc-8aa7-0cd3fbd43b80 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:30.998041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d76361-4808-4bfc-8aa7-0cd3fbd43b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:30.998314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d76361-4808-4bfc-8aa7-0cd3fbd43b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:30.999931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f6d76361-4808-4bfc-8aa7-0cd3fbd43b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:31.105799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d76361-4808-4bfc-8aa7-0cd3fbd43b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:31.107274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d76361-4808-4bfc-8aa7-0cd3fbd43b80 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:31.124954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 596/1186] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 06:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:59:31.469446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:59:31.470659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:59:31.482510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4217c-509b-4a25-bff6-13a0d9cceae9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:59:31.577512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:59:31.578141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:59:31.578414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:59:31.578678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] PATH_INFO: `/v3/projects/5e563923f9ee48cfa32fda247483bd80` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:59:31.589054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e563923f9ee48cfa32fda247483bd80)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:59:31.590125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:59:31.590125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 06:59:31.590125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 06:59:31.591347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:59:31.649771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e563923f9ee48cfa32fda247483bd80', 'request_id': 'req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:31.656307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e563923f9ee48cfa32fda247483bd80', 'request_id': 'req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:31.666727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e563923f9ee48cfa32fda247483bd80', 'request_id': 'req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:31.696138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] The token cache is being invalidate because project 5e563923f9ee48cfa32fda247483bd80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 06:59:31.696712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:31.696942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8c7d2d33-55f1-4c3d-82f3-4dce3e1cdee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 06:59:31.698942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 591/1187] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 06:59:31 2022] DELETE /identity/v3/projects/5e563923f9ee48cfa32fda247483bd80 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:59:41.122845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=62 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:59:41.694055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:00:59.293849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743b82-80ea-4706-b3c6-98386f375059 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:00:59.294433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743b82-80ea-4706-b3c6-98386f375059 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:00:59.294433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743b82-80ea-4706-b3c6-98386f375059 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:00:59.295796 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 597/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 07:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:00:59.300917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea328e-da7d-46f1-a7cd-ddcc104657f6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:00:59.301382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea328e-da7d-46f1-a7cd-ddcc104657f6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:00:59.301487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea328e-da7d-46f1-a7cd-ddcc104657f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:00:59.381208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dcea328e-da7d-46f1-a7cd-ddcc104657f6 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:00:59.382675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dcea328e-da7d-46f1-a7cd-ddcc104657f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:00:59.481903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 592/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 07:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:00:59.634660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67743b82-80ea-4706-b3c6-98386f375059 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:00:59.636793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-67743b82-80ea-4706-b3c6-98386f375059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:00:59.752925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67743b82-80ea-4706-b3c6-98386f375059 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:00:59.758296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd63328-da3e-4d68-9e1b-2eee25ed7998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:00:59.758889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd63328-da3e-4d68-9e1b-2eee25ed7998 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:00:59.759170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd63328-da3e-4d68-9e1b-2eee25ed7998 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:00:59.759469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd63328-da3e-4d68-9e1b-2eee25ed7998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:00:59.767341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd63328-da3e-4d68-9e1b-2eee25ed7998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:00:59.768915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd63328-da3e-4d68-9e1b-2eee25ed7998 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:00:59.778031 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 598/1190] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:00.514656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f576eaf-ae08-4665-af50-dd7755b33053 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:00.515241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f576eaf-ae08-4665-af50-dd7755b33053 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:00.515241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f576eaf-ae08-4665-af50-dd7755b33053 None None] PATH_INFO: `/v3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:00.516514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 593/1191] 10.222.0.33 () {54 vars in 925 bytes} [Mon Oct 17 07:01:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:01:00.523386 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494f322-56d1-460c-bf4d-2dd217fff53a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:00.523823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494f322-56d1-460c-bf4d-2dd217fff53a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:00.523958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494f322-56d1-460c-bf4d-2dd217fff53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:00.584631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5494f322-56d1-460c-bf4d-2dd217fff53a None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:00.585833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5494f322-56d1-460c-bf4d-2dd217fff53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:00.651092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:01:00.656779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 599/1192] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 17 07:01:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:00.861578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2e197-fea7-4532-ac66-33deafe38207 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:00.861781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2e197-fea7-4532-ac66-33deafe38207 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:00.862034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2e197-fea7-4532-ac66-33deafe38207 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:00.863425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 594/1193] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 07:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:01:00.870111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570da7-f6ff-47de-b9b1-dafdbf3db670 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:00.870389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570da7-f6ff-47de-b9b1-dafdbf3db670 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:00.870654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570da7-f6ff-47de-b9b1-dafdbf3db670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:00.890901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fb570da7-f6ff-47de-b9b1-dafdbf3db670 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:00.892142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fb570da7-f6ff-47de-b9b1-dafdbf3db670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:00.899910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 600/1194] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 07:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:01.053670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2e197-fea7-4532-ac66-33deafe38207 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:01.067534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2e197-fea7-4532-ac66-33deafe38207 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:01.073047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2c084-12c8-429f-b33e-e418260514f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:01.073628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c084-12c8-429f-b33e-e418260514f0 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:01.073899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c084-12c8-429f-b33e-e418260514f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:01.074161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c084-12c8-429f-b33e-e418260514f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:01.082441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c084-12c8-429f-b33e-e418260514f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:01.084041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c084-12c8-429f-b33e-e418260514f0 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:01.093593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 595/1195] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:09.225145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0c495-99a2-46e4-b8b6-c9bd943fe4fe None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:09.225623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0c495-99a2-46e4-b8b6-c9bd943fe4fe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:09.225725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0c495-99a2-46e4-b8b6-c9bd943fe4fe None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:09.227201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 601/1196] 10.222.0.36 () {54 vars in 961 bytes} [Mon Oct 17 07:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:01:09.232746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ad2d2-9273-4d83-9b5d-cd93017ad632 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:09.233183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ad2d2-9273-4d83-9b5d-cd93017ad632 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:09.233286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ad2d2-9273-4d83-9b5d-cd93017ad632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:09.253922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-559ad2d2-9273-4d83-9b5d-cd93017ad632 None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:09.255094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-559ad2d2-9273-4d83-9b5d-cd93017ad632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:09.263578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 596/1197] 10.222.0.36 () {58 vars in 1058 bytes} [Mon Oct 17 07:01:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:09.271116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0c495-99a2-46e4-b8b6-c9bd943fe4fe None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:09.284184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0c495-99a2-46e4-b8b6-c9bd943fe4fe None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:09.289069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefa876-9ca6-459e-96bb-57df6e6d45aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:09.289653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefa876-9ca6-459e-96bb-57df6e6d45aa service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:09.290062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefa876-9ca6-459e-96bb-57df6e6d45aa service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:09.290337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefa876-9ca6-459e-96bb-57df6e6d45aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:09.298761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefa876-9ca6-459e-96bb-57df6e6d45aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:09.300356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefa876-9ca6-459e-96bb-57df6e6d45aa service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:09.309646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 602/1198] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:12.554749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1da4-eac6-4a00-b7fd-e11f5350a482 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:12.555317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1da4-eac6-4a00-b7fd-e11f5350a482 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:12.555317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1da4-eac6-4a00-b7fd-e11f5350a482 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:12.556581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 597/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 07:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:01:12.562617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf884e4-3375-4537-bf0e-f99fd3f3231b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:12.563025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf884e4-3375-4537-bf0e-f99fd3f3231b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:12.563200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf884e4-3375-4537-bf0e-f99fd3f3231b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:12.582399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8bf884e4-3375-4537-bf0e-f99fd3f3231b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:12.583630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8bf884e4-3375-4537-bf0e-f99fd3f3231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:12.591937 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 603/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 07:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:12.745458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c1da4-eac6-4a00-b7fd-e11f5350a482 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:12.759733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c1da4-eac6-4a00-b7fd-e11f5350a482 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:12.765148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cdb21-9c45-4e2a-a658-182a7887d573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:12.765752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cdb21-9c45-4e2a-a658-182a7887d573 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:12.766029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cdb21-9c45-4e2a-a658-182a7887d573 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:12.766299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cdb21-9c45-4e2a-a658-182a7887d573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:12.774122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cdb21-9c45-4e2a-a658-182a7887d573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:12.775730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cdb21-9c45-4e2a-a658-182a7887d573 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:12.785556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 598/1201] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:13.849939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8718b5-a98e-4dda-8b10-854e9f93a23c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:13.850504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8718b5-a98e-4dda-8b10-854e9f93a23c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:13.850504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8718b5-a98e-4dda-8b10-854e9f93a23c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:13.851881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 604/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 07:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 07:01:13.858682 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8001f-c46b-40f4-9e2c-2a0646422568 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:13.859112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8001f-c46b-40f4-9e2c-2a0646422568 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:13.859242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8001f-c46b-40f4-9e2c-2a0646422568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:13.878982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-26b8001f-c46b-40f4-9e2c-2a0646422568 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:13.880197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-26b8001f-c46b-40f4-9e2c-2a0646422568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:13.887946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 599/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 07:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:14.039426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8718b5-a98e-4dda-8b10-854e9f93a23c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:14.053388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8718b5-a98e-4dda-8b10-854e9f93a23c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:14.058620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d49f-3104-4469-b483-241e6323689d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:14.059200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d49f-3104-4469-b483-241e6323689d service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:14.059502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d49f-3104-4469-b483-241e6323689d service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:14.059770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d49f-3104-4469-b483-241e6323689d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:14.068034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d49f-3104-4469-b483-241e6323689d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:14.069587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d49f-3104-4469-b483-241e6323689d service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:14.079351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 605/1204] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:18.017747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2926-ff6f-41c4-9691-b1efb8db6924 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:18.018345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2926-ff6f-41c4-9691-b1efb8db6924 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:18.018345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2926-ff6f-41c4-9691-b1efb8db6924 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:18.019647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 600/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 07:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:01:18.024701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b5159-e8a8-4c8c-8f3d-5a0beecea66a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:18.025156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b5159-e8a8-4c8c-8f3d-5a0beecea66a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:18.025260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b5159-e8a8-4c8c-8f3d-5a0beecea66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:18.045143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-338b5159-e8a8-4c8c-8f3d-5a0beecea66a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:18.046326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-338b5159-e8a8-4c8c-8f3d-5a0beecea66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:18.053991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 606/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 07:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:18.340722 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2926-ff6f-41c4-9691-b1efb8db6924 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:18.342957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b1eb2926-ff6f-41c4-9691-b1efb8db6924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:18.376401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:01:18.422170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2926-ff6f-41c4-9691-b1efb8db6924 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:18.425710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212668b2-7c5f-4f0e-aa46-4f964df39a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:18.426302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212668b2-7c5f-4f0e-aa46-4f964df39a9b service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:18.426576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212668b2-7c5f-4f0e-aa46-4f964df39a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:18.426845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212668b2-7c5f-4f0e-aa46-4f964df39a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:18.436022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212668b2-7c5f-4f0e-aa46-4f964df39a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:18.437453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212668b2-7c5f-4f0e-aa46-4f964df39a9b service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:18.446870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 601/1207] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:20.582007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af35803-a1df-4395-a670-76c44e4ce0c4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:20.582552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af35803-a1df-4395-a670-76c44e4ce0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:20.582552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af35803-a1df-4395-a670-76c44e4ce0c4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:20.583923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 607/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 07:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:01:20.589051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea85b0-75cd-4db8-9e7e-5966c9d48bac None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:20.589466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea85b0-75cd-4db8-9e7e-5966c9d48bac None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:20.589597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea85b0-75cd-4db8-9e7e-5966c9d48bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:20.609291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f8ea85b0-75cd-4db8-9e7e-5966c9d48bac None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:20.610490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f8ea85b0-75cd-4db8-9e7e-5966c9d48bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:20.618256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 602/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 07:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:20.906027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af35803-a1df-4395-a670-76c44e4ce0c4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:20.916552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af35803-a1df-4395-a670-76c44e4ce0c4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:20.919973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d543719-b039-4ebe-a382-369e6a282f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:20.920553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543719-b039-4ebe-a382-369e6a282f76 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:20.920833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543719-b039-4ebe-a382-369e6a282f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:20.921103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543719-b039-4ebe-a382-369e6a282f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:20.929582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d543719-b039-4ebe-a382-369e6a282f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:20.930987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d543719-b039-4ebe-a382-369e6a282f76 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:20.940655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 608/1210] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:24.514723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a965851-7b40-4984-9633-208797822f5d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:24.515273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a965851-7b40-4984-9633-208797822f5d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:24.515273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a965851-7b40-4984-9633-208797822f5d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:24.516543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 603/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 07:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:01:24.521623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448b9f1-856a-4fdd-934d-27997c777acb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:24.522032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448b9f1-856a-4fdd-934d-27997c777acb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:24.522149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448b9f1-856a-4fdd-934d-27997c777acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:24.541852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7448b9f1-856a-4fdd-934d-27997c777acb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:24.543084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7448b9f1-856a-4fdd-934d-27997c777acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:24.551439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 609/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 07:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:24.852614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a965851-7b40-4984-9633-208797822f5d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:24.854243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a965851-7b40-4984-9633-208797822f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:24.932429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a965851-7b40-4984-9633-208797822f5d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:24.936593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ad816-b3d0-453f-8ad8-cb1c686cc152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:24.937192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad816-b3d0-453f-8ad8-cb1c686cc152 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:24.937472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad816-b3d0-453f-8ad8-cb1c686cc152 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:24.937746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad816-b3d0-453f-8ad8-cb1c686cc152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:24.947097 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ad816-b3d0-453f-8ad8-cb1c686cc152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:24.948554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ad816-b3d0-453f-8ad8-cb1c686cc152 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:24.957787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 604/1213] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:27.074739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8902c-4a70-43f4-8d7d-e74f02a03df8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:27.075295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8902c-4a70-43f4-8d7d-e74f02a03df8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:27.075295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8902c-4a70-43f4-8d7d-e74f02a03df8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:27.077127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 610/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 07:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:01:27.082257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f272c-4dcb-4779-9e4f-efcca0c9a40f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:27.082696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f272c-4dcb-4779-9e4f-efcca0c9a40f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:27.082833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f272c-4dcb-4779-9e4f-efcca0c9a40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:27.101913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fd2f272c-4dcb-4779-9e4f-efcca0c9a40f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:01:27.103097 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fd2f272c-4dcb-4779-9e4f-efcca0c9a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:27.111292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 605/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 07:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:01:27.416845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8902c-4a70-43f4-8d7d-e74f02a03df8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:01:27.418497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f9b8902c-4a70-43f4-8d7d-e74f02a03df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:01:27.431155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8902c-4a70-43f4-8d7d-e74f02a03df8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:01:27.434573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688aa6e-d6a7-47a8-b1c8-1f4c0f8fa748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:01:27.435708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688aa6e-d6a7-47a8-b1c8-1f4c0f8fa748 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:01:27.436010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688aa6e-d6a7-47a8-b1c8-1f4c0f8fa748 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:01:27.436274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688aa6e-d6a7-47a8-b1c8-1f4c0f8fa748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:01:27.444762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688aa6e-d6a7-47a8-b1c8-1f4c0f8fa748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:01:27.446418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688aa6e-d6a7-47a8-b1c8-1f4c0f8fa748 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:01:27.455992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 611/1216] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:01:37.097204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=63 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:01:37.451947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=37 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:50.759225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57ffdc-ea7b-4abe-be45-7e1f62d4e5df None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:50.759862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57ffdc-ea7b-4abe-be45-7e1f62d4e5df None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:50.759862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57ffdc-ea7b-4abe-be45-7e1f62d4e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:50.782169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6b57ffdc-ea7b-4abe-be45-7e1f62d4e5df None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:50.783390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6b57ffdc-ea7b-4abe-be45-7e1f62d4e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:50.795358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc6ab3-0300-48f6-9913-8989b47692f5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:50.796357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc6ab3-0300-48f6-9913-8989b47692f5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:50.796875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc6ab3-0300-48f6-9913-8989b47692f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:50.797653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 606/1217] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:50.824722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3afc6ab3-0300-48f6-9913-8989b47692f5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:50.826491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3afc6ab3-0300-48f6-9913-8989b47692f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:50.832351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02998f8e-3adf-4f70-a749-e331015588bf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:50.832627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02998f8e-3adf-4f70-a749-e331015588bf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:50.832899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02998f8e-3adf-4f70-a749-e331015588bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:50.840187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 612/1218] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 17 07:02:50.859890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-02998f8e-3adf-4f70-a749-e331015588bf None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:50.861080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-02998f8e-3adf-4f70-a749-e331015588bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:50.870172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35d177-3b01-4d17-b194-932ff0a0aec5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:50.870770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35d177-3b01-4d17-b194-932ff0a0aec5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:50.871294 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35d177-3b01-4d17-b194-932ff0a0aec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:50.879345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 607/1219] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:50.883505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02998f8e-3adf-4f70-a749-e331015588bf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:50.897010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02998f8e-3adf-4f70-a749-e331015588bf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:50.901642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:50.906754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:50.906754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:50.906754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:50.906754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:50.906754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:50.906754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:50.909063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ee35d177-3b01-4d17-b194-932ff0a0aec5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:50.910724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ee35d177-3b01-4d17-b194-932ff0a0aec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:50.920949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 608/1220] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:02:50.929379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:50.938853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 613/1221] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:02:50.944139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35d177-3b01-4d17-b194-932ff0a0aec5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:50.959199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4b931-6fe9-4695-9ca9-37f8e93ec49f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:50.964992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35d177-3b01-4d17-b194-932ff0a0aec5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:50.970262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:50.970901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:50.971176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:50.971470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:50.972120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:50.973177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 07:02:50.973177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 07:02:50.973177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 07:02:50.974395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:50.980287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:50.980927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:50.981295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:50.981563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:50.982231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:50.982532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:50.983611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 07:02:50.983611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 07:02:50.983611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 07:02:50.984520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.016454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 614/1222] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:02:51.023747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.039249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28931c11-881d-4ba2-b476-e4f73f6d7519 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.050200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.050775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.051053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.051458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.054951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.058550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.064923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 609/1223] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:50 2022] POST /identity/v3/projects => generated 403 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:02:51.068594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.119622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 615/1224] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:02:51.124599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.186675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b19f4-51f1-4432-8c37-f70a83531cfe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.267739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0def9d-d032-4f46-b4b2-c53bdfbdf57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.268351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0def9d-d032-4f46-b4b2-c53bdfbdf57d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.268627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0def9d-d032-4f46-b4b2-c53bdfbdf57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.268896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0def9d-d032-4f46-b4b2-c53bdfbdf57d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.269755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0def9d-d032-4f46-b4b2-c53bdfbdf57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.271705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0def9d-d032-4f46-b4b2-c53bdfbdf57d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.274242 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7b90a-6446-4151-83c9-eeeaa812873f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.300873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1abe0-33a4-4eeb-b5e5-7417c34ae132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.301754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1abe0-33a4-4eeb-b5e5-7417c34ae132 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.302162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1abe0-33a4-4eeb-b5e5-7417c34ae132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.302480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1abe0-33a4-4eeb-b5e5-7417c34ae132 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.303150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1abe0-33a4-4eeb-b5e5-7417c34ae132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.307102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-8dc1abe0-33a4-4eeb-b5e5-7417c34ae132 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 07:02:51.307102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 07:02:51.307102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 07:02:51.309936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1abe0-33a4-4eeb-b5e5-7417c34ae132 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.311621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 610/1225] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/users => generated 302 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:02:51.315811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0c45a-9d6e-44b4-a5e5-d8869ccf3030 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.316095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0c45a-9d6e-44b4-a5e5-d8869ccf3030 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.316366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0c45a-9d6e-44b4-a5e5-d8869ccf3030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.344882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e0e0c45a-9d6e-44b4-a5e5-d8869ccf3030 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.346188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e0e0c45a-9d6e-44b4-a5e5-d8869ccf3030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.357787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 611/1226] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.358695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0c45a-9d6e-44b4-a5e5-d8869ccf3030 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.360148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 616/1227] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/users => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:02:51.361624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ce876-22e5-4082-a5e7-43f39aa13344 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.362136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ce876-22e5-4082-a5e7-43f39aa13344 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.362617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ce876-22e5-4082-a5e7-43f39aa13344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.373709 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0c45a-9d6e-44b4-a5e5-d8869ccf3030 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.380070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53b06e-6d08-47ad-b4ea-341102a34aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.380697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53b06e-6d08-47ad-b4ea-341102a34aad admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.380974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53b06e-6d08-47ad-b4ea-341102a34aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.381301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53b06e-6d08-47ad-b4ea-341102a34aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.381945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53b06e-6d08-47ad-b4ea-341102a34aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.383851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53b06e-6d08-47ad-b4ea-341102a34aad admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.388449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a07ce876-22e5-4082-a5e7-43f39aa13344 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.389630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a07ce876-22e5-4082-a5e7-43f39aa13344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.395190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 612/1228] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:51.396223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577243e-95ec-483d-971d-6cc08501eab5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.396505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577243e-95ec-483d-971d-6cc08501eab5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.396777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577243e-95ec-483d-971d-6cc08501eab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.398352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 617/1229] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.400025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdff6-3012-4685-833a-420c1d8f7fe2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.400538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdff6-3012-4685-833a-420c1d8f7fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.401068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdff6-3012-4685-833a-420c1d8f7fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.421416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b577243e-95ec-483d-971d-6cc08501eab5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.422960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b577243e-95ec-483d-971d-6cc08501eab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.424476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3fffdff6-3012-4685-833a-420c1d8f7fe2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.425667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3fffdff6-3012-4685-833a-420c1d8f7fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.432987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 613/1230] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.433470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 618/1231] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.435047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577243e-95ec-483d-971d-6cc08501eab5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.442569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cfcfc-3c50-42a3-9ee2-028ae331cf65 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.443166 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cfcfc-3c50-42a3-9ee2-028ae331cf65 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.443485 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cfcfc-3c50-42a3-9ee2-028ae331cf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.451326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577243e-95ec-483d-971d-6cc08501eab5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.456553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733cb0a-d014-4c60-8b63-e4f081b50265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.457207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733cb0a-d014-4c60-8b63-e4f081b50265 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.457486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733cb0a-d014-4c60-8b63-e4f081b50265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.457904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733cb0a-d014-4c60-8b63-e4f081b50265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.458536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733cb0a-d014-4c60-8b63-e4f081b50265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.460178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733cb0a-d014-4c60-8b63-e4f081b50265 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.464835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8c8cfcfc-3c50-42a3-9ee2-028ae331cf65 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.466535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8c8cfcfc-3c50-42a3-9ee2-028ae331cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.470865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 614/1232] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:51.472149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ddc7-b70d-4510-9e54-eb13061a000c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.472429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ddc7-b70d-4510-9e54-eb13061a000c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.472729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ddc7-b70d-4510-9e54-eb13061a000c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.476603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 619/1233] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.477574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cfcfc-3c50-42a3-9ee2-028ae331cf65 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.484387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:51.487926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cfcfc-3c50-42a3-9ee2-028ae331cf65 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.492547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd88b90-0730-456f-b244-e57d0808711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.493130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88b90-0730-456f-b244-e57d0808711c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.493410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88b90-0730-456f-b244-e57d0808711c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.493691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88b90-0730-456f-b244-e57d0808711c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.494313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd88b90-0730-456f-b244-e57d0808711c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.495936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd88b90-0730-456f-b244-e57d0808711c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.497141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8a00ddc7-b70d-4510-9e54-eb13061a000c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.498355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a00ddc7-b70d-4510-9e54-eb13061a000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.506055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 620/1234] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:51.514090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 615/1235] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.514276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f826d1f-3a35-44a1-846f-c10a1d9781d1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.514276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f826d1f-3a35-44a1-846f-c10a1d9781d1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.514439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2cee7-731c-4d34-9037-b69ec6861e3a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.514526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f826d1f-3a35-44a1-846f-c10a1d9781d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.514623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2cee7-731c-4d34-9037-b69ec6861e3a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.514623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2cee7-731c-4d34-9037-b69ec6861e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.531694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d0e2cee7-731c-4d34-9037-b69ec6861e3a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.532023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7f826d1f-3a35-44a1-846f-c10a1d9781d1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.533011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d0e2cee7-731c-4d34-9037-b69ec6861e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.533207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7f826d1f-3a35-44a1-846f-c10a1d9781d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.541037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 621/1236] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.541393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 616/1237] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.542611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69734-6286-4f44-9df2-cc0064d263f5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.542892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69734-6286-4f44-9df2-cc0064d263f5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.543160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69734-6286-4f44-9df2-cc0064d263f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.546432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1c1c-d136-4e4d-aeb1-34801234b2ac None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.546719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1c1c-d136-4e4d-aeb1-34801234b2ac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.546993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1c1c-d136-4e4d-aeb1-34801234b2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.564721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-74e69734-6286-4f44-9df2-cc0064d263f5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.566374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-74e69734-6286-4f44-9df2-cc0064d263f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.571760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-101e1c1c-d136-4e4d-aeb1-34801234b2ac None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.573209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-101e1c1c-d136-4e4d-aeb1-34801234b2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.574813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 617/1238] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.575647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e69734-6286-4f44-9df2-cc0064d263f5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.585462 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e69734-6286-4f44-9df2-cc0064d263f5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.586283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 622/1239] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.587259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1c1c-d136-4e4d-aeb1-34801234b2ac None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.592114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.592724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.593003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.593280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.593922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.595580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.597539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1c1c-d136-4e4d-aeb1-34801234b2ac None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.602081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f01a0-614a-4dd1-9953-705a6ca50130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.602665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f01a0-614a-4dd1-9953-705a6ca50130 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.602943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f01a0-614a-4dd1-9953-705a6ca50130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.603227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f01a0-614a-4dd1-9953-705a6ca50130 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.606866 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 618/1240] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:51.608000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.609527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f01a0-614a-4dd1-9953-705a6ca50130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:51.609841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f01a0-614a-4dd1-9953-705a6ca50130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.613307 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f01a0-614a-4dd1-9953-705a6ca50130 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.622420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230ae2-9cb6-4327-80fb-99cd45d1bfb3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.626693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 623/1241] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:02:51.627964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17aab3e-79d7-4da3-9a66-365392ec449b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.628241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17aab3e-79d7-4da3-9a66-365392ec449b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.628506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17aab3e-79d7-4da3-9a66-365392ec449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.634502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.635171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.635474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.635751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] PATH_INFO: `/v3/projects/1f328f45cc4e4fd2b96e948d4a5e0f77/users/6d81321bfeb1403ba9753943d9db4f12/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.647515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e17aab3e-79d7-4da3-9a66-365392ec449b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.648793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e17aab3e-79d7-4da3-9a66-365392ec449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.650508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f328f45cc4e4fd2b96e948d4a5e0f77, user_id=6d81321bfeb1403ba9753943d9db4f12, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.653810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.657289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 624/1242] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.658120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17aab3e-79d7-4da3-9a66-365392ec449b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.667554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17aab3e-79d7-4da3-9a66-365392ec449b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.668036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 619/1243] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:51 2022] PUT /identity/v3/projects/1f328f45cc4e4fd2b96e948d4a5e0f77/users/6d81321bfeb1403ba9753943d9db4f12/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:51.672770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.700220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:51.724387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad8b21-875c-4eac-8b21-24cb01740eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.724998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad8b21-875c-4eac-8b21-24cb01740eda admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.725285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad8b21-875c-4eac-8b21-24cb01740eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.725557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad8b21-875c-4eac-8b21-24cb01740eda admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.726221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad8b21-875c-4eac-8b21-24cb01740eda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:51.726523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad8b21-875c-4eac-8b21-24cb01740eda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.728150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad8b21-875c-4eac-8b21-24cb01740eda admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.733852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774cc2c9-4a89-4974-96a5-69e674f39d62 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.738396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.738995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.739310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.739587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.740001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 625/1244] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:02:51.741250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3600e7-91e9-46be-835b-0e2bc3046660 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.741534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3600e7-91e9-46be-835b-0e2bc3046660 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.741809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3600e7-91e9-46be-835b-0e2bc3046660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.743546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:51.743933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.745758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.763454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 620/1245] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:02:51.763454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.764443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6d3600e7-91e9-46be-835b-0e2bc3046660 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.765641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6d3600e7-91e9-46be-835b-0e2bc3046660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.773136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cddb17-a62f-46ab-8b57-bc54877c9eb6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.777324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 626/1246] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.777324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3600e7-91e9-46be-835b-0e2bc3046660 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.781225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.781815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.782088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.782355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.782976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:51.783302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.784839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3600e7-91e9-46be-835b-0e2bc3046660 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.784964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.789391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.790020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.790297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.790567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] PATH_INFO: `/v3/projects/a6f1df8f7c6a4b449e3030ddc8aa9f70/users/84f3d2f808744afa895f1b7f00963fa3/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.797440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 621/1247] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:02:51.798328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.806852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f1df8f7c6a4b449e3030ddc8aa9f70, user_id=84f3d2f808744afa895f1b7f00963fa3, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.807603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d329-142f-439a-a462-8ffac37db443 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.809576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.812168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484e810-8360-4fb4-9531-15eafc7b7ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.812752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e810-8360-4fb4-9531-15eafc7b7ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.813031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e810-8360-4fb4-9531-15eafc7b7ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.813303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e810-8360-4fb4-9531-15eafc7b7ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.813923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484e810-8360-4fb4-9531-15eafc7b7ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.815915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484e810-8360-4fb4-9531-15eafc7b7ef1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.824230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 627/1248] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:51 2022] PUT /identity/v3/projects/a6f1df8f7c6a4b449e3030ddc8aa9f70/users/84f3d2f808744afa895f1b7f00963fa3/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:51.825933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.846383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 622/1249] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:51.846383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31169b-336e-45a3-b25a-4b2488baf4bd None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.846383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31169b-336e-45a3-b25a-4b2488baf4bd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.848812 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31169b-336e-45a3-b25a-4b2488baf4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.864844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eb31169b-336e-45a3-b25a-4b2488baf4bd None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:51.866063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eb31169b-336e-45a3-b25a-4b2488baf4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:51.890305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef321a-da7a-4d3f-b1ec-38b2abb06003 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.923668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 623/1250] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:51.924713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31169b-336e-45a3-b25a-4b2488baf4bd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.936369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31169b-336e-45a3-b25a-4b2488baf4bd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.941131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.941727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.942005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.942278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.942904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.944872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.946583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.947183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.947483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.948018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.948672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:51.948967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.951044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:51.951177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:51.963822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 628/1251] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:02:51.971269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.974438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6645af7-da16-4477-8390-2e4631b7f370 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:51.981698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 624/1252] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:51.981698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:51.981951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:51.981951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:51.981951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:51.981951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:51.982338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:51.984151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.011562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 629/1253] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.012991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.033211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:52.045200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fad13-d9f5-4c5c-aea7-3fd3c71f66c6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.076212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981d397-47da-4c2d-8929-e8fd53115305 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.080935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd99f5-724c-4b7c-9a32-5fa007c7838d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.081536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd99f5-724c-4b7c-9a32-5fa007c7838d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.081812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd99f5-724c-4b7c-9a32-5fa007c7838d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.082079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd99f5-724c-4b7c-9a32-5fa007c7838d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.082695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd99f5-724c-4b7c-9a32-5fa007c7838d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.084641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd99f5-724c-4b7c-9a32-5fa007c7838d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.102612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.103365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.103759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.104039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.104733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:52.105209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.106927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.111888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 630/1254] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.113251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ed268-9a46-429c-84e7-850685188b3d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.113536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ed268-9a46-429c-84e7-850685188b3d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.113816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ed268-9a46-429c-84e7-850685188b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.127486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 625/1255] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:02:52.127486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.132033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824273a-12f6-4474-ad7d-fd167440e8d4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.142680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-065ed268-9a46-429c-84e7-850685188b3d None None] MFA Rules not processed for user `84f3d2f808744afa895f1b7f00963fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:52.143944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-065ed268-9a46-429c-84e7-850685188b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:52.190176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.190807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.191078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.191373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.191990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.193882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.211083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 631/1256] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:52.211965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ed268-9a46-429c-84e7-850685188b3d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.218953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 626/1257] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.219637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.230909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f623-a9a1-4c97-bce0-63829187abd7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.237467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.238054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.238332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.238603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.239283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.241156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.256866 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:52.269264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 627/1258] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.272636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ed268-9a46-429c-84e7-850685188b3d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.272758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.320554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.321161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.321438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.321708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.326112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:52.333614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143fd3f-2ddb-4f74-a569-5d9f460e9426 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.338038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.338640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.338903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.339168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.339819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.342381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.366989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 628/1259] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.366989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.374556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106618-7107-4eb1-af41-5a3a2c2c877c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.379028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.379642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.379913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.380204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.380816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.382735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.385437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.386865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.396556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 632/1260] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:52.397402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.405381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 629/1261] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.406171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.407201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b8d6-d124-41e0-8efc-7573c5b75828 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.412365 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.413400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.413400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.413681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.414300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.416240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.417444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452e8f8-cdf4-4d42-ad76-2027bcb792e7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.423819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.424489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.424766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.425034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.425897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.429736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.439272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 633/1262] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.443587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.454724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa8be7-d117-4289-afeb-f99ea0fff192 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.463805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 630/1263] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.463805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.466523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.467092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.467407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.467703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.468322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.470285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.505837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 634/1264] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/projects => generated 401 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.505837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.536936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19830c3e-4581-480d-8263-d8ec35106968 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.578771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9d543-264b-4356-9a2b-024fce2bdebe admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.592542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.593224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:52.594361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.594644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.595053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.595897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.600686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.608542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.609369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.609645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.609962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.610604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.612262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.624747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 635/1265] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.625379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 631/1266] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.626175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.629979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.637802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b424ba-c215-40fa-95d6-81f6fb523d42 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.640684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611af116-c7ea-4012-b6ad-bec06e043202 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.642409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.643062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.643380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.643702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.644328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.645554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.645911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.646157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.646434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.646708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.647365 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.648957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.656740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 632/1267] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.657525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.663819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 636/1268] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.664988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.670899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa89d-ab4c-4fe7-bc10-fba232050036 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.676059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa24de9-d2df-477a-aaac-558c2179a6a2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.676772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.677426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.677703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.677970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.678597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.680231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.680994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.681757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.682035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.682648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.683298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.685239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.694102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 633/1269] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.694941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.707559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f02bc-7572-4e6d-9cc5-7262f3ccc386 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.712953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 637/1270] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.714039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.715837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.719245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.719377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.719676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.720664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.723078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.728074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df9edd-b026-40ac-85f7-62e71d9a163d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.734558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.735784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.736264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.737091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.737945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.740261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.751868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 634/1271] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:52 2022] POST /identity/v3/users => generated 307 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:52.751868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.756475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 638/1272] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.756475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.764315 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9852b-b73b-4d59-9a65-7ef7e7ce460f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.765958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384f8c-f8af-43c8-b041-ed6ecd3c085c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.770337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.770942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.771227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.771467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.771609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.772268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.772380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.772711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.773078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.773809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.773925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.775814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.782273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:52.785519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 635/1273] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.786511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.791384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 639/1274] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.792561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.798929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8390fc-48c6-4274-81d2-68d75d9beb7e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.803747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8b3ff-7727-4828-99fd-0a259b0333b0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.803945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.804658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.804936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.805206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.805831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.807449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.808613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.809284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.809569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.809841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.810471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.812088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.820336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 636/1275] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.820994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.823309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 640/1276] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.824148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.833090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a58df-d871-49f1-8b95-0d44e809fc8d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.837898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e112c1-981f-4c0c-aa46-5041641a3948 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.838637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.839529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.839968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.840455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.841301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.843035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.843660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.844449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.844824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.845177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.845923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.847737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.855847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 637/1277] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.857208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.862571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 641/1278] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.863632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.868708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184fe1-d331-423e-994f-b28bf6a0e912 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.875130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.875832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.876349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.876561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] PATH_INFO: `/v3/projects/0cb1aa55658342fcaf2f493f51e9faeb/users/c987a5a52fc641f78054ac0218edcd13/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.877706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b990d0-18c5-4863-a43b-edeb0692e26b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.883260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.884139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.884625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.885052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.885925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.888502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.895004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb1aa55658342fcaf2f493f51e9faeb, user_id=c987a5a52fc641f78054ac0218edcd13, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:52.897729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:52.902217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 642/1279] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:52.903253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.913932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 638/1280] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:52 2022] PUT /identity/v3/projects/0cb1aa55658342fcaf2f493f51e9faeb/users/c987a5a52fc641f78054ac0218edcd13/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:52.914864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408629f-e413-4dc3-9c98-f98caf373d57 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.915172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:52.983154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.983866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.984142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.984410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] PATH_INFO: `/v3/projects/63468386eaae4369b44f37a4235217c3/users/0863a3df7ae840608827c1865c3a784c/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:52.992786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79063-ecd5-4346-95f5-f7c97dfab69f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:52.998236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:52.999422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:52.999422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:52.999778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] PATH_INFO: `/v3/projects/f992a8ccca8d487abd91dc6d0baaf3bc/users/51b2f292cea24a21928e2adcef0c3786/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.015953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=63468386eaae4369b44f37a4235217c3, user_id=0863a3df7ae840608827c1865c3a784c, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.018738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.022703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f992a8ccca8d487abd91dc6d0baaf3bc, user_id=51b2f292cea24a21928e2adcef0c3786, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.025959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.034720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 643/1281] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:52 2022] PUT /identity/v3/projects/63468386eaae4369b44f37a4235217c3/users/0863a3df7ae840608827c1865c3a784c/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:53.035671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.058934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 639/1282] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:52 2022] PUT /identity/v3/projects/f992a8ccca8d487abd91dc6d0baaf3bc/users/51b2f292cea24a21928e2adcef0c3786/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:53.060420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.104199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:53.106224 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9e70e-ed75-4167-ba74-cc0ecf124cfe admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.135953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598954-2e04-46c7-ba78-8d7e36575a5e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.142666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.143474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.144040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.144416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.145171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.147487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.160335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 640/1283] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:53.164326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.176552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a761-5655-4fe4-94ba-d1c667417847 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.182011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.182937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.183198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.183869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] PATH_INFO: `/v3/projects/38501e5b190740adb4ec3ad873998869/users/3e7ec752b2904006b258d7fa17daceb5/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.188603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:53.197273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5eb7d-04ea-447e-af04-51f856ac27d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.199689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5eb7d-04ea-447e-af04-51f856ac27d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.200899 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5eb7d-04ea-447e-af04-51f856ac27d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.201330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5eb7d-04ea-447e-af04-51f856ac27d1 admin admin] PATH_INFO: `/v3/projects/934764e1a1394bfb91688060b594a58a/users/1e37d9ed3751483ebe4141706f46e43a/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.204610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38501e5b190740adb4ec3ad873998869, user_id=3e7ec752b2904006b258d7fa17daceb5, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.208039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.220601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5eb7d-04ea-447e-af04-51f856ac27d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934764e1a1394bfb91688060b594a58a, user_id=1e37d9ed3751483ebe4141706f46e43a, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.223856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5eb7d-04ea-447e-af04-51f856ac27d1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.227408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 641/1284] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:53 2022] PUT /identity/v3/projects/38501e5b190740adb4ec3ad873998869/users/3e7ec752b2904006b258d7fa17daceb5/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:53.228486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.247470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 644/1285] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:53 2022] PUT /identity/v3/projects/934764e1a1394bfb91688060b594a58a/users/1e37d9ed3751483ebe4141706f46e43a/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:53.250025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702780c9-379f-4467-89b0-11cd9de6865f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.250407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702780c9-379f-4467-89b0-11cd9de6865f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.250576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702780c9-379f-4467-89b0-11cd9de6865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.271171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-702780c9-379f-4467-89b0-11cd9de6865f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:53.272241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-702780c9-379f-4467-89b0-11cd9de6865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.311391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e76389-ea99-4547-b9be-87e66739793c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.339451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 645/1286] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:53.341179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490b64-55c8-4cc9-a83d-3b942df83616 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.341543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490b64-55c8-4cc9-a83d-3b942df83616 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.341798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490b64-55c8-4cc9-a83d-3b942df83616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.375332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-46490b64-55c8-4cc9-a83d-3b942df83616 None None] MFA Rules not processed for user `0863a3df7ae840608827c1865c3a784c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:53.376045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-46490b64-55c8-4cc9-a83d-3b942df83616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.378973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18242a5e-b220-4150-bf98-31da80583814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.379680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242a5e-b220-4150-bf98-31da80583814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.380249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242a5e-b220-4150-bf98-31da80583814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.380530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242a5e-b220-4150-bf98-31da80583814 admin admin] PATH_INFO: `/v3/projects/9c8dd2001d1b4a44ad171b014dcffc68/users/de82574fa84a4ef58565cb06f61f7a68/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.396559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18242a5e-b220-4150-bf98-31da80583814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8dd2001d1b4a44ad171b014dcffc68, user_id=de82574fa84a4ef58565cb06f61f7a68, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.399259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18242a5e-b220-4150-bf98-31da80583814 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.412694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 642/1287] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:53 2022] PUT /identity/v3/projects/9c8dd2001d1b4a44ad171b014dcffc68/users/de82574fa84a4ef58565cb06f61f7a68/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:53.423163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4f62-8ad1-4d5f-a5de-fb1db52d84c6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.423163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4f62-8ad1-4d5f-a5de-fb1db52d84c6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.423163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4f62-8ad1-4d5f-a5de-fb1db52d84c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.434759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b2ed4f62-8ad1-4d5f-a5de-fb1db52d84c6 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:53.435986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b2ed4f62-8ad1-4d5f-a5de-fb1db52d84c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.463549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 646/1288] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:53.464544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490b64-55c8-4cc9-a83d-3b942df83616 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.468169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:53.476436 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490b64-55c8-4cc9-a83d-3b942df83616 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.494912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 643/1289] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:53.496046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4f62-8ad1-4d5f-a5de-fb1db52d84c6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.505772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4f62-8ad1-4d5f-a5de-fb1db52d84c6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.510257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7e5ed-164a-4c97-ae65-a281feabcdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.510842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e5ed-164a-4c97-ae65-a281feabcdec admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.511116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e5ed-164a-4c97-ae65-a281feabcdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.511415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e5ed-164a-4c97-ae65-a281feabcdec admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.512037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e5ed-164a-4c97-ae65-a281feabcdec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.513659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e5ed-164a-4c97-ae65-a281feabcdec admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.523478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 644/1290] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:53.524602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.524889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.525156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.540980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7121dae-ee10-426a-b5d8-a7900b651824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.541646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7121dae-ee10-426a-b5d8-a7900b651824 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.541926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7121dae-ee10-426a-b5d8-a7900b651824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.542205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7121dae-ee10-426a-b5d8-a7900b651824 admin admin] PATH_INFO: `/v3/projects/b07dcfcb56324cc3a64ade8b763ae573/users/8d144842b98e477aaef275278ad8db7e/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.553266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] MFA Rules not processed for user `6d81321bfeb1403ba9753943d9db4f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:53.554502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.558265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7121dae-ee10-426a-b5d8-a7900b651824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07dcfcb56324cc3a64ade8b763ae573, user_id=8d144842b98e477aaef275278ad8db7e, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.561565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7121dae-ee10-426a-b5d8-a7900b651824 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.563732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:53.576568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 647/1291] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:53 2022] PUT /identity/v3/projects/b07dcfcb56324cc3a64ade8b763ae573/users/8d144842b98e477aaef275278ad8db7e/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:53.578117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb898cc1-0875-43c9-b400-332a5a3fa230 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.578608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb898cc1-0875-43c9-b400-332a5a3fa230 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.579077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb898cc1-0875-43c9-b400-332a5a3fa230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.598123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-eb898cc1-0875-43c9-b400-332a5a3fa230 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:53.599362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eb898cc1-0875-43c9-b400-332a5a3fa230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.622388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 645/1292] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:53.623255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.624507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.660259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 648/1293] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:53.661624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90321dc6-845b-4b0c-bb8b-8ad95dce1b5c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.661922 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90321dc6-845b-4b0c-bb8b-8ad95dce1b5c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.662197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90321dc6-845b-4b0c-bb8b-8ad95dce1b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.680455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093fcb3-a515-4e6f-bf88-f424058651b3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.691328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-90321dc6-845b-4b0c-bb8b-8ad95dce1b5c None None] MFA Rules not processed for user `de82574fa84a4ef58565cb06f61f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:53.692734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-90321dc6-845b-4b0c-bb8b-8ad95dce1b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.728386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.728989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.729262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.729526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.739027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.740494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.749917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 646/1294] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:53.750688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.759490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 649/1295] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:53.764112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c3d46-6ac7-41d1-9529-079de455988d service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.764228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90321dc6-845b-4b0c-bb8b-8ad95dce1b5c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.768149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d10d8-ffb4-4fc1-83f6-8e01d8f0c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.768824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d10d8-ffb4-4fc1-83f6-8e01d8f0c4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.769183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d10d8-ffb4-4fc1-83f6-8e01d8f0c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.769516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d10d8-ffb4-4fc1-83f6-8e01d8f0c4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.770206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d10d8-ffb4-4fc1-83f6-8e01d8f0c4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.772029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90321dc6-845b-4b0c-bb8b-8ad95dce1b5c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.773092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d10d8-ffb4-4fc1-83f6-8e01d8f0c4c8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.776367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.777174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.778535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.778535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.792894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:53.795929 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.797655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.810201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 650/1296] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:53.811759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.813765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 647/1297] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:53.815087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7441c84-1818-4c58-b1c7-c5f0ec34d20e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.815417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7441c84-1818-4c58-b1c7-c5f0ec34d20e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.815690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7441c84-1818-4c58-b1c7-c5f0ec34d20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.823671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74661f6c-b7c0-4dca-bf1c-9df33cac1f4e service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:53.841287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e7441c84-1818-4c58-b1c7-c5f0ec34d20e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:53.842934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e7441c84-1818-4c58-b1c7-c5f0ec34d20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:53.897516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:53.899311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:53.899698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:53.900318 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] PATH_INFO: `/v3/projects/38501e5b190740adb4ec3ad873998869/users/3e7ec752b2904006b258d7fa17daceb5/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:53.903320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=38501e5b190740adb4ec3ad873998869, user_id=3e7ec752b2904006b258d7fa17daceb5, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:53.906939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:53.918916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 648/1298] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:53.918916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7441c84-1818-4c58-b1c7-c5f0ec34d20e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.921853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 651/1299] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:53 2022] PUT /identity/v3/projects/38501e5b190740adb4ec3ad873998869/users/3e7ec752b2904006b258d7fa17daceb5/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:53.922994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:53.929135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:53.934952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7441c84-1818-4c58-b1c7-c5f0ec34d20e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.004547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09942f1-eb36-4a1e-b772-729a0e8929ae admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.008858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5862daa-50f9-451c-911e-06fb7ff30746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.009246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5862daa-50f9-451c-911e-06fb7ff30746 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.009485 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5862daa-50f9-451c-911e-06fb7ff30746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.009790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5862daa-50f9-451c-911e-06fb7ff30746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.013379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecb5e6-0de0-418a-b98e-4b3efc7b830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.014420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecb5e6-0de0-418a-b98e-4b3efc7b830e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.014700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecb5e6-0de0-418a-b98e-4b3efc7b830e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.015155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecb5e6-0de0-418a-b98e-4b3efc7b830e admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.016103 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecb5e6-0de0-418a-b98e-4b3efc7b830e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.018314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecb5e6-0de0-418a-b98e-4b3efc7b830e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.025673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5862daa-50f9-451c-911e-06fb7ff30746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.027901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5862daa-50f9-451c-911e-06fb7ff30746 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.038773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 652/1300] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:54.040295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5aa28-8938-45c7-8bba-10ede0ad7ecd None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.040599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5aa28-8938-45c7-8bba-10ede0ad7ecd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.040912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5aa28-8938-45c7-8bba-10ede0ad7ecd None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.042248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 653/1301] 10.222.0.36 () {54 vars in 961 bytes} [Mon Oct 17 07:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:02:54.042955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5aa28-8938-45c7-8bba-10ede0ad7ecd None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.054167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 649/1302] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:53 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:54.055402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efe26-654d-4f4a-ae24-299ea1caf323 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.055678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efe26-654d-4f4a-ae24-299ea1caf323 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.055956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efe26-654d-4f4a-ae24-299ea1caf323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.065530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5aa28-8938-45c7-8bba-10ede0ad7ecd None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.095446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b38efe26-654d-4f4a-ae24-299ea1caf323 None None] MFA Rules not processed for user `3e7ec752b2904006b258d7fa17daceb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:54.097346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b38efe26-654d-4f4a-ae24-299ea1caf323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:54.154157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.155566 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.155951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.157167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.158596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.164728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.204075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 654/1303] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:54 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:54.205153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.242080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 650/1304] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:54 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:54.244398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2869fb-fa54-4bc6-ac2f-412cb264d25d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.245051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2869fb-fa54-4bc6-ac2f-412cb264d25d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.245897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2869fb-fa54-4bc6-ac2f-412cb264d25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.263955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:54.297162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfdc9e-0b12-4073-824f-9559334d34dc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.302309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.303659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.304165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.304882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.316550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5e2869fb-fa54-4bc6-ac2f-412cb264d25d None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:54.317948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5e2869fb-fa54-4bc6-ac2f-412cb264d25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:54.321547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.324097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.338797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 655/1305] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:54.339969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.355652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7acd-cac8-4665-9908-9e4e15addfd8 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.374094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.375814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.376258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.376371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.377042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.379837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.425354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 656/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:54 2022] POST /identity/v3/users => generated 309 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:02:54.426472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.449014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c05f71-6cff-4706-8961-6426fd38f6a2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.460021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.461042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.461329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.461599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.463722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.465096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 651/1307] 10.222.0.36 () {58 vars in 1058 bytes} [Mon Oct 17 07:02:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:54.467875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2869fb-fa54-4bc6-ac2f-412cb264d25d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.468894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.504810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 657/1308] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:54.505993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.525272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dfd76-088f-4b67-aaf1-131526033312 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.538920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2869fb-fa54-4bc6-ac2f-412cb264d25d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.545057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.545848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.546121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.546399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.548703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.549768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.550309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.551929 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.552979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.555804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.558931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.561227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.566152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:54.576752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 658/1309] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:54.578245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.579492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 652/1310] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:54 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:54.580705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.595433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dadb50-ef35-4714-86eb-8926fd4c044e service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.596023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d825a5a-4c0b-4d63-8609-761df6a22faa admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.603508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.604300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.604808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.605667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.607467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.608172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.608548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.608883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] PATH_INFO: `/v3/projects/067c5a629ef344d8ba723b9c88433640/users/0177790fdf0f4feeb415aadbfe4d45f6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.643241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=067c5a629ef344d8ba723b9c88433640, user_id=0177790fdf0f4feeb415aadbfe4d45f6, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.646356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.681544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 653/1311] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:54 2022] PUT /identity/v3/projects/067c5a629ef344d8ba723b9c88433640/users/0177790fdf0f4feeb415aadbfe4d45f6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:54.681544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.691946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.694335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.705429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 659/1312] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:54.706052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.718331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e47fcd-fc7d-4787-baef-a933e1ad3a17 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.831192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e832d-80a1-404d-b8c3-84645db0ca24 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:54.847691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.849126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.849414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.849691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] PATH_INFO: `/v3/projects/6f1c30d21c1c4a67838e717b1ac5e1e0/users/c0866699304047bb80033e6747f24982/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.854721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:54.856991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:54.858015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:54.858920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:54.859973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:54.861578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.872200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.890716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1c30d21c1c4a67838e717b1ac5e1e0, user_id=c0866699304047bb80033e6747f24982, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:54.894794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:54.905992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 660/1313] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:02:54.909237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.924868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 654/1314] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:54 2022] PUT /identity/v3/projects/6f1c30d21c1c4a67838e717b1ac5e1e0/users/c0866699304047bb80033e6747f24982/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:54.928826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:54.941490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b15694-3cd1-4105-9609-e1f1760ebb56 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:55.079431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85c8a0-75cc-4f75-a78b-6ad0275aac66 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:55.081203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:55.082034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.082949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.084163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] PATH_INFO: `/v3/projects/067c5a629ef344d8ba723b9c88433640/users/0177790fdf0f4feeb415aadbfe4d45f6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.085775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:55.086546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.086827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.087097 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.087758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:55.088450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=067c5a629ef344d8ba723b9c88433640, user_id=0177790fdf0f4feeb415aadbfe4d45f6, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:55.090201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:55.096145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:55.108538 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 655/1315] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:02:55.109594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:55.123764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 661/1316] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:54 2022] PUT /identity/v3/projects/067c5a629ef344d8ba723b9c88433640/users/0177790fdf0f4feeb415aadbfe4d45f6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:55.125507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:55.139510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d98a29-cab8-4093-a243-1892848d3d83 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:55.193970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:55.255398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:55.256748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.257270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.257891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] PATH_INFO: `/v3/users/84f3d2f808744afa895f1b7f00963fa3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.259852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f3d2f808744afa895f1b7f00963fa3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:55.259976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c68a7-917f-4dbd-ab23-04d01684b3d5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:55.262201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 07:02:55.262201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Oct 17 07:02:55.262201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: Oct 17 07:02:55.266110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c8e7-8e95-4e2f-bbfc-e842d1a05b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:55.266597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c8e7-8e95-4e2f-bbfc-e842d1a05b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.266997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:55.267189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c8e7-8e95-4e2f-bbfc-e842d1a05b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.267424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c8e7-8e95-4e2f-bbfc-e842d1a05b18 admin admin] PATH_INFO: `/v3/projects/6f1c30d21c1c4a67838e717b1ac5e1e0/users/c0866699304047bb80033e6747f24982/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.269996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c8e7-8e95-4e2f-bbfc-e842d1a05b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1c30d21c1c4a67838e717b1ac5e1e0, user_id=c0866699304047bb80033e6747f24982, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:55.272772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c8e7-8e95-4e2f-bbfc-e842d1a05b18 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:55.285871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 662/1317] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:02:55 2022] PUT /identity/v3/projects/6f1c30d21c1c4a67838e717b1ac5e1e0/users/c0866699304047bb80033e6747f24982/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:55.287388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06836c-a827-4997-81d8-5cdf888e6ec5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.287719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06836c-a827-4997-81d8-5cdf888e6ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.287994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06836c-a827-4997-81d8-5cdf888e6ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.321162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6a06836c-a827-4997-81d8-5cdf888e6ec5 None None] MFA Rules not processed for user `0177790fdf0f4feeb415aadbfe4d45f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:55.322494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6a06836c-a827-4997-81d8-5cdf888e6ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:55.389799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f3d2f808744afa895f1b7f00963fa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:55.391251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f3d2f808744afa895f1b7f00963fa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:55.413010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f3d2f808744afa895f1b7f00963fa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:55.425706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f3d2f808744afa895f1b7f00963fa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:55.426484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74aae690-8952-41b7-847e-d282f923fdbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f3d2f808744afa895f1b7f00963fa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:55.438481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 663/1318] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:55.444910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06836c-a827-4997-81d8-5cdf888e6ec5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:55.474810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:55.492841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 656/1319] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:02:55 2022] DELETE /identity/v3/users/84f3d2f808744afa895f1b7f00963fa3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:55.496175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f6ab-6004-43d2-acdc-71a1a053a786 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.497939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f6ab-6004-43d2-acdc-71a1a053a786 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.498291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f6ab-6004-43d2-acdc-71a1a053a786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.547587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4aa8f6ab-6004-43d2-acdc-71a1a053a786 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:55.548953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4aa8f6ab-6004-43d2-acdc-71a1a053a786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:55.579243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06836c-a827-4997-81d8-5cdf888e6ec5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:55.651853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:55.652525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.652847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.654038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.656176 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:55.658189 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 657/1320] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:55.677463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:55.753749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:55.756213 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:55.772894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 664/1321] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:55.829058 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521d5da-e813-43d0-88f1-141dd0266a70 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.829341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521d5da-e813-43d0-88f1-141dd0266a70 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.829620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521d5da-e813-43d0-88f1-141dd0266a70 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.831635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 658/1322] 10.222.0.36 () {54 vars in 961 bytes} [Mon Oct 17 07:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:02:55.851655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:55.853337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:55.857582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493d91c-3b31-46b2-abff-6e1c1bcec4dc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.857875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493d91c-3b31-46b2-abff-6e1c1bcec4dc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.858149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493d91c-3b31-46b2-abff-6e1c1bcec4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.897278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1493d91c-3b31-46b2-abff-6e1c1bcec4dc None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:55.898656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1493d91c-3b31-46b2-abff-6e1c1bcec4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:55.946524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bfa34-51af-4de8-83ba-0f4921ebc707 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:55.952023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:55.952663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:55.952927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:55.953196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:55.966511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:55.968443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:55.979249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 665/1323] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:55.980112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:55.981780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:55.998998 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 659/1324] 10.222.0.36 () {58 vars in 1058 bytes} [Mon Oct 17 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:02:56.004798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754471-9731-406e-b42f-270443c3ecb3 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.005156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754471-9731-406e-b42f-270443c3ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.005496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754471-9731-406e-b42f-270443c3ecb3 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.007258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 660/1325] 10.222.0.36 () {54 vars in 912 bytes} [Mon Oct 17 07:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:02:56.012441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.012787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.013257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.015851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc2801-541a-4d72-a990-b794c8fae29d service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:56.025403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc58ac4-4757-406c-879d-3cc5d5896a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:56.026026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc58ac4-4757-406c-879d-3cc5d5896a37 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.026304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc58ac4-4757-406c-879d-3cc5d5896a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.026576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc58ac4-4757-406c-879d-3cc5d5896a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.033078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5fc58ac4-4757-406c-879d-3cc5d5896a37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:56.038322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:56.040585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:56.068630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 661/1326] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:02:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:02:56.092242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:56.095244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:56.114584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d359-eda4-42d1-8794-a18f81b048c3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:56.130809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a33eb-8fab-4a11-80c2-e2833064460a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:56.132435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a33eb-8fab-4a11-80c2-e2833064460a service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.134295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a33eb-8fab-4a11-80c2-e2833064460a service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.134769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a33eb-8fab-4a11-80c2-e2833064460a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.162813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc58ac4-4757-406c-879d-3cc5d5896a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:56.171104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc58ac4-4757-406c-879d-3cc5d5896a37 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:56.177666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a33eb-8fab-4a11-80c2-e2833064460a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:56.183553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a33eb-8fab-4a11-80c2-e2833064460a service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:56.228867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 666/1327] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:56.263344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 662/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:56.421149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213079-04e2-4dd4-8de4-205308394aa5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.422270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213079-04e2-4dd4-8de4-205308394aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.422733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213079-04e2-4dd4-8de4-205308394aa5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.424930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 667/1329] 10.222.0.36 () {54 vars in 912 bytes} [Mon Oct 17 07:02:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:02:56.440634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.441512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.442050 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.567916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:56.567916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:56.567916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 663/1330] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:02:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:56.570522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e213079-04e2-4dd4-8de4-205308394aa5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:56.572576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e213079-04e2-4dd4-8de4-205308394aa5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:56.579641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1e471-f7ab-4ae6-bb94-f789abcef85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:56.580298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e471-f7ab-4ae6-bb94-f789abcef85b service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.580791 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e471-f7ab-4ae6-bb94-f789abcef85b service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.581079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e471-f7ab-4ae6-bb94-f789abcef85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.604446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1e471-f7ab-4ae6-bb94-f789abcef85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:56.606415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1e471-f7ab-4ae6-bb94-f789abcef85b service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:56.646221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 668/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:56.803336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:56.804791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:56.828174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d70b2-2dd6-45b4-b5e8-3b55434a2e16 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:56.833330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:56.834183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.834669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.835017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] PATH_INFO: `/v3/users/de82574fa84a4ef58565cb06f61f7a68` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:56.836423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de82574fa84a4ef58565cb06f61f7a68)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:56.839377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:56.901384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de82574fa84a4ef58565cb06f61f7a68'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:56.902114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de82574fa84a4ef58565cb06f61f7a68'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:56.912190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de82574fa84a4ef58565cb06f61f7a68'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:56.917778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:56.942452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de82574fa84a4ef58565cb06f61f7a68'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:56.943809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de82574fa84a4ef58565cb06f61f7a68'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:56.986063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 664/1332] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:02:56 2022] DELETE /identity/v3/users/de82574fa84a4ef58565cb06f61f7a68 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:56.990655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:56.991147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:56.991279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:57.029995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:57.031754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:57.132551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:57.182653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 669/1333] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:57.230192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:57.232419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:57.337710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099556-e4b2-453f-bf94-39479f214325 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:57.342492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:57.343122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:57.343422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:57.343698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:57.355107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:57.357339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:57.374449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 665/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:02:57.529821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:57.535202 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:57.554759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880882b-a3ba-47f2-95da-20eb5abeb627 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:57.571535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:57.572435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:57.572553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:57.572988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] PATH_INFO: `/v3/projects/a6f1df8f7c6a4b449e3030ddc8aa9f70` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:57.579866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f1df8f7c6a4b449e3030ddc8aa9f70)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:57.581079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 07:02:57.581079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Oct 17 07:02:57.581079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: Oct 17 07:02:57.582291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:57.688967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f1df8f7c6a4b449e3030ddc8aa9f70', 'request_id': 'req-4e4aaf40-84b0-4444-94ef-8e886412eb5d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:57.699134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f1df8f7c6a4b449e3030ddc8aa9f70', 'request_id': 'req-4e4aaf40-84b0-4444-94ef-8e886412eb5d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:57.717700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f1df8f7c6a4b449e3030ddc8aa9f70', 'request_id': 'req-4e4aaf40-84b0-4444-94ef-8e886412eb5d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:57.738286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] The token cache is being invalidate because project a6f1df8f7c6a4b449e3030ddc8aa9f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:02:57.739754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:57.740894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4e4aaf40-84b0-4444-94ef-8e886412eb5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:57.744488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 670/1335] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:02:57 2022] DELETE /identity/v3/projects/a6f1df8f7c6a4b449e3030ddc8aa9f70 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:57.749217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:57.762243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd11b-1ec8-435f-9400-81a9e382b9cd service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:57.889373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:57.890887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:57.891455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:57.891957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] PATH_INFO: `/v3/users/3e7ec752b2904006b258d7fa17daceb5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:57.894417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7ec752b2904006b258d7fa17daceb5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:57.898199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:58.021039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7ec752b2904006b258d7fa17daceb5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:58.024332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7ec752b2904006b258d7fa17daceb5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:58.039236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7ec752b2904006b258d7fa17daceb5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:58.056996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7ec752b2904006b258d7fa17daceb5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:58.058631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7ec752b2904006b258d7fa17daceb5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:58.089145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:58.101707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 666/1336] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:02:57 2022] DELETE /identity/v3/users/3e7ec752b2904006b258d7fa17daceb5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:02:59.011849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1c853-2756-4472-ad2c-b8565c019b86 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:59.012356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1c853-2756-4472-ad2c-b8565c019b86 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:59.015117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1c853-2756-4472-ad2c-b8565c019b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:59.048391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b0c1c853-2756-4472-ad2c-b8565c019b86 None None] MFA Rules not processed for user `c0866699304047bb80033e6747f24982`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:59.049724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b0c1c853-2756-4472-ad2c-b8565c019b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:59.058355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:59.062328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:59.175690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 671/1337] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:59.240140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb68607-0863-4d97-8918-32c5156a4918 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:59.258432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:59.259855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:59.260407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:59.261650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] PATH_INFO: `/v3/projects/9c8dd2001d1b4a44ad171b014dcffc68` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:59.267798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c8dd2001d1b4a44ad171b014dcffc68)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:59.276751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:59.395425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c8dd2001d1b4a44ad171b014dcffc68', 'request_id': 'req-47394eac-3bed-46c9-ab19-f24c38f89081'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:59.407376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c8dd2001d1b4a44ad171b014dcffc68', 'request_id': 'req-47394eac-3bed-46c9-ab19-f24c38f89081'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:59.423396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c8dd2001d1b4a44ad171b014dcffc68', 'request_id': 'req-47394eac-3bed-46c9-ab19-f24c38f89081'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:59.443706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] The token cache is being invalidate because project 9c8dd2001d1b4a44ad171b014dcffc68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:02:59.443706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:59.444436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47394eac-3bed-46c9-ab19-f24c38f89081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:02:59.445975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 667/1338] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:02:59 2022] DELETE /identity/v3/projects/9c8dd2001d1b4a44ad171b014dcffc68 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:02:59.452870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae821a-ea80-4abd-b454-9cd89883e227 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:59.453268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae821a-ea80-4abd-b454-9cd89883e227 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:59.454114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae821a-ea80-4abd-b454-9cd89883e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:59.499034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f4ae821a-ea80-4abd-b454-9cd89883e227 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:59.500547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f4ae821a-ea80-4abd-b454-9cd89883e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:59.620385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:02:59.667457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf50f6-366b-483c-889c-1a0edcc9232f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:59.667697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 672/1339] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:02:59.667809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf50f6-366b-483c-889c-1a0edcc9232f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:59.667993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf50f6-366b-483c-889c-1a0edcc9232f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:59.712454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-55bf50f6-366b-483c-889c-1a0edcc9232f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:02:59.713820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-55bf50f6-366b-483c-889c-1a0edcc9232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:02:59.732977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 668/1340] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:02:59.737260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae821a-ea80-4abd-b454-9cd89883e227 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:59.759072 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae821a-ea80-4abd-b454-9cd89883e227 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:59.771714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:59.773272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:59.773272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:59.775731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:59.778360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:02:59.780924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:59.785323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:59.818046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 673/1341] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:02:59.818755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf50f6-366b-483c-889c-1a0edcc9232f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:59.831288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf50f6-366b-483c-889c-1a0edcc9232f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:02:59.852540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:02:59.853116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:02:59.853399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:02:59.853655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:02:59.854321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:02:59.856478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:02:59.916635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 669/1342] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:02:59 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:02:59.920956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:02:59.935989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841fb99-7eb0-47a5-87c8-081158b1aaa1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:00.118027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:00.119471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.119911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.120335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.121230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:00.125805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:00.176424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 674/1343] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:02:59 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:00.180242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:00.192126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26901e3b-cc15-48cc-9272-85aeebdb1e76 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:00.214824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:00.218849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.219149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.219630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.220336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:00.223362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:00.253836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 670/1344] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:00.256847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:00.290702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56859b3-2a6a-453e-8b3e-8c195293dc3d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:00.325911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:00.328345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:00.328530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.329619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.330204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:00.331172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.338523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:00.349867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:00.350389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e0c0-9055-4f22-a52c-8c5f78c02279 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:00.364236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:00.365904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.366814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.367099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] PATH_INFO: `/v3/projects/38501e5b190740adb4ec3ad873998869` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.371048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38501e5b190740adb4ec3ad873998869)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:00.375856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:00.459084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 675/1345] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:00.469574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:00.490693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:00.502004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38501e5b190740adb4ec3ad873998869', 'request_id': 'req-12548beb-bce4-4d83-abe7-5606401a551a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:00.515412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38501e5b190740adb4ec3ad873998869', 'request_id': 'req-12548beb-bce4-4d83-abe7-5606401a551a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:00.530765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38501e5b190740adb4ec3ad873998869', 'request_id': 'req-12548beb-bce4-4d83-abe7-5606401a551a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:00.535845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2928a80-3300-4904-a0b5-2767fec58c38 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:00.564222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] The token cache is being invalidate because project 38501e5b190740adb4ec3ad873998869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:00.564528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:00.566046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12548beb-bce4-4d83-abe7-5606401a551a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:00.571514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 671/1346] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:00 2022] DELETE /identity/v3/projects/38501e5b190740adb4ec3ad873998869 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:00.576229 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa137997-0ac9-442c-b861-a57392bfae1d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.576665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa137997-0ac9-442c-b861-a57392bfae1d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.576942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa137997-0ac9-442c-b861-a57392bfae1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.639126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fa137997-0ac9-442c-b861-a57392bfae1d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:00.644083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fa137997-0ac9-442c-b861-a57392bfae1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:00.767366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa60625-454f-4f38-940c-2c68aae2b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:00.768124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60625-454f-4f38-940c-2c68aae2b45d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.768404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60625-454f-4f38-940c-2c68aae2b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.768678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60625-454f-4f38-940c-2c68aae2b45d admin admin] PATH_INFO: `/v3/projects/629158ad423c4e6c90af6bf991fa4831/users/65a69676a2bf48cbbee53f6c112e37bc/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.802384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa60625-454f-4f38-940c-2c68aae2b45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=629158ad423c4e6c90af6bf991fa4831, user_id=65a69676a2bf48cbbee53f6c112e37bc, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:00.805723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa60625-454f-4f38-940c-2c68aae2b45d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:00.834892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 676/1347] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:00 2022] PUT /identity/v3/projects/629158ad423c4e6c90af6bf991fa4831/users/65a69676a2bf48cbbee53f6c112e37bc/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:00.847202 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.847613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.847816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.872434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 672/1348] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:00.883807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702beeb-92a4-45d1-b92d-9b0721280200 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:00.884578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702beeb-92a4-45d1-b92d-9b0721280200 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:00.885308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702beeb-92a4-45d1-b92d-9b0721280200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:00.906635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] MFA Rules not processed for user `65a69676a2bf48cbbee53f6c112e37bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:00.907915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:00.951028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c702beeb-92a4-45d1-b92d-9b0721280200 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:00.953322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c702beeb-92a4-45d1-b92d-9b0721280200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:00.977377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:01.049769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 677/1349] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:01.054946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.061796 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:01.075506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 673/1350] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:01.077000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702beeb-92a4-45d1-b92d-9b0721280200 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.091169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702beeb-92a4-45d1-b92d-9b0721280200 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.097270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.098188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.098473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.098746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.099953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:01.100267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.101700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8d9bb-d531-4e09-b99a-e902f01886c2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.107094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.126947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.127758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.128116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.128715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.130455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.132532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.137914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 674/1351] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:01.139095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.171320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287615ec-426d-438b-9e5c-ae242e8e51c4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.184809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.186118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.186408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.186935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.188292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.194626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.196388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 678/1352] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:01 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:01.200240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.255561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 675/1353] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:01 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:01.256444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.343356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db503f04-501f-4c5b-949a-2b8bc4c62e8d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.358906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:01.410958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3d344-3358-4f6c-b0aa-c6cfe45d2066 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.423978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.426464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.427306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.428643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.429373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.433333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.487430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 676/1354] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:01 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:01.494068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.497968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.498847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.499686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.500549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.502141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.506575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.517701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f0333-628b-4632-8026-f399aecc356c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.534520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.537869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.538919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.539877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.540962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.551128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.574618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 679/1355] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:01 2022] POST /identity/v3/users => generated 295 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:01.579509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.595400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 677/1356] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:01.596627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.624414 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2495-b9a2-437b-a3ae-9e8dbed4e6b7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.642038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c390a20-2fed-4f08-ba2c-577dc98a9d8a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.646790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74025c-b4bd-4a4f-a60f-1de773770c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.647875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74025c-b4bd-4a4f-a60f-1de773770c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.650133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74025c-b4bd-4a4f-a60f-1de773770c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.651489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74025c-b4bd-4a4f-a60f-1de773770c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.655656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74025c-b4bd-4a4f-a60f-1de773770c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.660699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74025c-b4bd-4a4f-a60f-1de773770c25 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.666861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.671913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.674596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.677694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.678492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.680997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.702712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 680/1357] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:01.702712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e88cde-2cd1-4697-9366-e8a8a3f5af12 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.702712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e88cde-2cd1-4697-9366-e8a8a3f5af12 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.702712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e88cde-2cd1-4697-9366-e8a8a3f5af12 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.706952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 681/1358] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:03:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:03:01.707787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e88cde-2cd1-4697-9366-e8a8a3f5af12 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.718950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 678/1359] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:01.721974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:01.744514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e88cde-2cd1-4697-9366-e8a8a3f5af12 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.745948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6067b-e3d1-4380-9965-1fde248a4c89 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:01.757508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa7eb1-f316-4743-b28f-7c595f945ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.761010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7eb1-f316-4743-b28f-7c595f945ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.762746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7eb1-f316-4743-b28f-7c595f945ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.765841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7eb1-f316-4743-b28f-7c595f945ab9 admin admin] PATH_INFO: `/v3/projects/2bef9510338246e899acbf71e4a57adf/users/f38c62f644af4853bdad65cd464c9b69/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.784997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:01.785307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.785688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.786150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] PATH_INFO: `/v3/projects/e6317c258bda49789a5ab783dfd0dedc/users/dee6061675c547ccb231083fc7228bc1/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.814649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa7eb1-f316-4743-b28f-7c595f945ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bef9510338246e899acbf71e4a57adf, user_id=f38c62f644af4853bdad65cd464c9b69, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.818675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa7eb1-f316-4743-b28f-7c595f945ab9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.862060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 682/1360] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:01 2022] PUT /identity/v3/projects/2bef9510338246e899acbf71e4a57adf/users/f38c62f644af4853bdad65cd464c9b69/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:01.862060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df626-194f-4848-a07b-a9c71b3a30ee None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:01.862060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df626-194f-4848-a07b-a9c71b3a30ee None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:01.862060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df626-194f-4848-a07b-a9c71b3a30ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:01.889089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6317c258bda49789a5ab783dfd0dedc, user_id=dee6061675c547ccb231083fc7228bc1, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:01.894522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:01.920006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4f9df626-194f-4848-a07b-a9c71b3a30ee None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:01.924322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4f9df626-194f-4848-a07b-a9c71b3a30ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:01.960477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:01.976597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 679/1361] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:01 2022] PUT /identity/v3/projects/e6317c258bda49789a5ab783dfd0dedc/users/dee6061675c547ccb231083fc7228bc1/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:01.977764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:02.132980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 683/1362] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:03:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:02.134317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c52b-dea6-4078-b8ad-e744c3eebcf3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:02.134669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c52b-dea6-4078-b8ad-e744c3eebcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:02.135142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c52b-dea6-4078-b8ad-e744c3eebcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:02.235051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6500c52b-dea6-4078-b8ad-e744c3eebcf3 None None] MFA Rules not processed for user `dee6061675c547ccb231083fc7228bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:02.236299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6500c52b-dea6-4078-b8ad-e744c3eebcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:02.318567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae81ab5-eea3-4542-bd23-ef6d6250515a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:02.333210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:02.333836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:02.334108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:02.334375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:02.335065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:02.339127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:02.377933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 680/1363] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:02.378833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:02.385474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:02.588863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 684/1364] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:02.590536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500c52b-dea6-4078-b8ad-e744c3eebcf3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:02.647443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340de91-e17e-4ca9-95be-da7da97203fc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:02.648544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500c52b-dea6-4078-b8ad-e744c3eebcf3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:02.656018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:02.656658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:02.656936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:02.657309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:02.673243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6121c6a-ab69-4654-b9f4-0695f756cac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:02.675118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121c6a-ab69-4654-b9f4-0695f756cac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:02.675118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121c6a-ab69-4654-b9f4-0695f756cac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:02.675503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121c6a-ab69-4654-b9f4-0695f756cac1 admin admin] PATH_INFO: `/v3/projects/2bef9510338246e899acbf71e4a57adf/users/f38c62f644af4853bdad65cd464c9b69/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:02.682770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:02.686365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:02.689049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6121c6a-ab69-4654-b9f4-0695f756cac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bef9510338246e899acbf71e4a57adf, user_id=f38c62f644af4853bdad65cd464c9b69, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:02.704626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6121c6a-ab69-4654-b9f4-0695f756cac1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:02.714031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 681/1365] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:02.716121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:02.747723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 685/1366] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:02 2022] PUT /identity/v3/projects/2bef9510338246e899acbf71e4a57adf/users/f38c62f644af4853bdad65cd464c9b69/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:02.755671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:02.756094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:02.756578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:02.765261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273a789-ce1c-44c1-abd1-f2dfa84de4b4 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:02.783362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:02.894186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] MFA Rules not processed for user `f38c62f644af4853bdad65cd464c9b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:02.896476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:02.958150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:02.958150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:02.958150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:02.958150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:02.966132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:02.979544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:03.051149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 682/1367] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:02 2022] POST /identity/v3/projects => generated 373 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:03.054705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:03.235474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3172b-6543-449e-b970-4ed57c0c4ce3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:03.241107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:03.241715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:03.241986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:03.242251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:03.243139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:03.245196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:03.282102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 683/1368] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:03 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:03.283372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:03.291570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 686/1369] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 538 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:03.296667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d23c6-a9b1-4f55-a03b-faddc122632a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:03.303062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:03.305909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:03.306220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:03.307132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:03.307459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:03.307947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:03.308599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:03.310322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:03.330717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 684/1370] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:03.335168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:03.335912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:03.360327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af517-3d47-4f44-86f9-10a9e5a60de9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:03.379797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:03.380697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:03.381909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:03.381909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] PATH_INFO: `/v3/projects/a29e4301528e443ca019de9de38d52da/users/d44529781ca847bc8d57f99d38a3bee2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:03.476490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29e4301528e443ca019de9de38d52da, user_id=d44529781ca847bc8d57f99d38a3bee2, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:03.482905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:03.496477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232777ff-56ab-4cf9-bd36-cdfdaf8cbaf4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:03.510165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 685/1371] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:03 2022] PUT /identity/v3/projects/a29e4301528e443ca019de9de38d52da/users/d44529781ca847bc8d57f99d38a3bee2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:03.514232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:03.628789 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803f8bc-9c57-4404-98aa-042206777bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:03.629419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803f8bc-9c57-4404-98aa-042206777bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:03.629695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803f8bc-9c57-4404-98aa-042206777bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:03.631363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803f8bc-9c57-4404-98aa-042206777bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:03.642098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:03.673009 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d33f77-3a05-404e-bf3e-5487614cb16e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:03.687897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:03.689042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:03.689375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:03.689645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:03.690539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:03.693866 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:03.726443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 686/1372] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:03.739555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:03.758743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803f8bc-9c57-4404-98aa-042206777bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:03.760448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803f8bc-9c57-4404-98aa-042206777bf8 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:03.762262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0cd28-d001-4b7a-b88e-dd3a7b013182 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:03.768161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:03.771348 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:03.771683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:03.772023 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] PATH_INFO: `/v3/projects/a29e4301528e443ca019de9de38d52da/users/d44529781ca847bc8d57f99d38a3bee2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:03.774631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29e4301528e443ca019de9de38d52da, user_id=d44529781ca847bc8d57f99d38a3bee2, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:03.777211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:03.792130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 687/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:03.795628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 687/1374] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:03 2022] PUT /identity/v3/projects/a29e4301528e443ca019de9de38d52da/users/d44529781ca847bc8d57f99d38a3bee2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:03.806585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b9ea-88fa-41cd-83a4-49eeb7240a1b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:03.807862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b9ea-88fa-41cd-83a4-49eeb7240a1b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:03.808319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b9ea-88fa-41cd-83a4-49eeb7240a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:03.875232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e2d5b9ea-88fa-41cd-83a4-49eeb7240a1b None None] MFA Rules not processed for user `d44529781ca847bc8d57f99d38a3bee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:03.876885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e2d5b9ea-88fa-41cd-83a4-49eeb7240a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:04.052020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 688/1375] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:04.060738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:04.062978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:04.268420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232670c6-3310-444e-9e2e-aa0e9c2accf7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:04.280338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff88903-247d-4660-a1d4-bf7dd608c345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:04.282486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff88903-247d-4660-a1d4-bf7dd608c345 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:04.284373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff88903-247d-4660-a1d4-bf7dd608c345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:04.284373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff88903-247d-4660-a1d4-bf7dd608c345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:04.320617 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff88903-247d-4660-a1d4-bf7dd608c345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:04.322111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff88903-247d-4660-a1d4-bf7dd608c345 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:04.364757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 688/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:04.653817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5b9ea-88fa-41cd-83a4-49eeb7240a1b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:04.659502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:04.672396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5b9ea-88fa-41cd-83a4-49eeb7240a1b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:04.679747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0390d2-c9c4-45b8-b111-6e6a6ac27e34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:04.680374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0390d2-c9c4-45b8-b111-6e6a6ac27e34 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:04.680651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0390d2-c9c4-45b8-b111-6e6a6ac27e34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:04.680920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0390d2-c9c4-45b8-b111-6e6a6ac27e34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:04.835450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0390d2-c9c4-45b8-b111-6e6a6ac27e34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:04.836926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0390d2-c9c4-45b8-b111-6e6a6ac27e34 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:04.854730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 689/1377] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:05.307190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277af79-2da4-46d9-a429-8b36f6f00ee5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:05.307785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277af79-2da4-46d9-a429-8b36f6f00ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:05.308137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277af79-2da4-46d9-a429-8b36f6f00ee5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:05.311244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 689/1378] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:03:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:03:05.359359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d613-9c0a-4dbc-8b04-e773f7e6ef53 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:05.359647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d613-9c0a-4dbc-8b04-e773f7e6ef53 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:05.360141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d613-9c0a-4dbc-8b04-e773f7e6ef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:05.384409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277af79-2da4-46d9-a429-8b36f6f00ee5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:05.386871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8277af79-2da4-46d9-a429-8b36f6f00ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:05.411269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5258d613-9c0a-4dbc-8b04-e773f7e6ef53 None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:05.421967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5258d613-9c0a-4dbc-8b04-e773f7e6ef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:05.603257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277af79-2da4-46d9-a429-8b36f6f00ee5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:05.616486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:05.617192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:05.617472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:05.618137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] PATH_INFO: `/v3/users/6d81321bfeb1403ba9753943d9db4f12` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:05.620072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d81321bfeb1403ba9753943d9db4f12)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:05.622440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:05.639173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 690/1379] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:03:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:05.760687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d81321bfeb1403ba9753943d9db4f12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:05.764167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d81321bfeb1403ba9753943d9db4f12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:05.806687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d81321bfeb1403ba9753943d9db4f12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:05.827624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d81321bfeb1403ba9753943d9db4f12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:05.828681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e721bed4-f9d1-4956-bf0a-1bbdc8b53011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d81321bfeb1403ba9753943d9db4f12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:05.850085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:05.880566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 690/1380] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:05 2022] DELETE /identity/v3/users/6d81321bfeb1403ba9753943d9db4f12 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:05.910432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258d613-9c0a-4dbc-8b04-e773f7e6ef53 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:05.923351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258d613-9c0a-4dbc-8b04-e773f7e6ef53 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:06.081266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:06.082089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:06.082480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:06.082857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:06.085553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:06.100006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:06.101736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:06.138053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 691/1381] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:07.560374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501c67-6d59-45be-8db8-b378201d03ee None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:07.560904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501c67-6d59-45be-8db8-b378201d03ee None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:07.566006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501c67-6d59-45be-8db8-b378201d03ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:07.615339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:07.617592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:07.639914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ee501c67-6d59-45be-8db8-b378201d03ee None None] MFA Rules not processed for user `c987a5a52fc641f78054ac0218edcd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:07.644291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ee501c67-6d59-45be-8db8-b378201d03ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:07.728919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:07.857003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf00655-9813-4238-adba-99fe048fcb2d service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:07.873428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:07.874272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:07.874598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:07.874941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] PATH_INFO: `/v3/projects/1f328f45cc4e4fd2b96e948d4a5e0f77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:07.876807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f328f45cc4e4fd2b96e948d4a5e0f77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:07.889533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:07.904645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 691/1382] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:07.905974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:07.906490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:07.906620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:07.941666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] MFA Rules not processed for user `c987a5a52fc641f78054ac0218edcd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:07.942909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:07.972704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 692/1383] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:07.974800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f328f45cc4e4fd2b96e948d4a5e0f77', 'request_id': 'req-d4814706-5b14-48bf-8c1c-681cd19710c6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:07.981861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f328f45cc4e4fd2b96e948d4a5e0f77', 'request_id': 'req-d4814706-5b14-48bf-8c1c-681cd19710c6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:07.983753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:07.986163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:08.007030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f328f45cc4e4fd2b96e948d4a5e0f77', 'request_id': 'req-d4814706-5b14-48bf-8c1c-681cd19710c6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.039119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] The token cache is being invalidate because project 1f328f45cc4e4fd2b96e948d4a5e0f77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:08.039825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.049948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.056114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 692/1384] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:07 2022] DELETE /identity/v3/projects/1f328f45cc4e4fd2b96e948d4a5e0f77 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:08.056901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:08.058441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:08.209599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae16c0d-0449-49e5-845a-92555496b0f1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:08.229229 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4814706-5b14-48bf-8c1c-681cd19710c6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:08.248271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:08.249090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:08.249487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:08.249873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] PATH_INFO: `/v3/users/c0866699304047bb80033e6747f24982` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:08.251285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0866699304047bb80033e6747f24982)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:08.253759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:08.362893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0866699304047bb80033e6747f24982'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.364082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0866699304047bb80033e6747f24982'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.379843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0866699304047bb80033e6747f24982'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.381950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0866699304047bb80033e6747f24982'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.395513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:08.438388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42dd6f3b-915f-4cbd-beb5-42ef5f468bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0866699304047bb80033e6747f24982'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:08.443153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19d9b5-33ae-4912-b70a-a44ce494dc5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:08.443810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19d9b5-33ae-4912-b70a-a44ce494dc5f service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:08.444088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19d9b5-33ae-4912-b70a-a44ce494dc5f service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:08.444362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19d9b5-33ae-4912-b70a-a44ce494dc5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:08.451337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9c19d9b5-33ae-4912-b70a-a44ce494dc5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:08.485449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 693/1385] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:08 2022] DELETE /identity/v3/users/c0866699304047bb80033e6747f24982 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:08.487148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:08.639937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19d9b5-33ae-4912-b70a-a44ce494dc5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:08.644365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19d9b5-33ae-4912-b70a-a44ce494dc5f service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:08.676038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 693/1386] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:10.003994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6e5a-b31e-4ba8-975a-45858c6bfb9b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:10.004877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6e5a-b31e-4ba8-975a-45858c6bfb9b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:10.005303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6e5a-b31e-4ba8-975a-45858c6bfb9b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:10.006848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 694/1387] 10.222.0.36 () {54 vars in 912 bytes} [Mon Oct 17 07:03:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:03:10.019539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627a86d-19bd-4ab2-81d8-07226fd30c21 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:10.019539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627a86d-19bd-4ab2-81d8-07226fd30c21 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:10.020099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627a86d-19bd-4ab2-81d8-07226fd30c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:10.145020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8627a86d-19bd-4ab2-81d8-07226fd30c21 None None] MFA Rules not processed for user `56967a5531ea4e308af5b89c13524c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:10.146990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8627a86d-19bd-4ab2-81d8-07226fd30c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:10.179055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6e5a-b31e-4ba8-975a-45858c6bfb9b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:10.181051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b3ff6e5a-b31e-4ba8-975a-45858c6bfb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:10.361623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 694/1388] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:03:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:10.434934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6e5a-b31e-4ba8-975a-45858c6bfb9b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:10.456364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:10.457056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:10.457338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:10.457610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] PATH_INFO: `/v3/projects/6f1c30d21c1c4a67838e717b1ac5e1e0` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:10.461682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f1c30d21c1c4a67838e717b1ac5e1e0)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:10.468815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:10.601810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f1c30d21c1c4a67838e717b1ac5e1e0', 'request_id': 'req-9d1a8acd-45d5-4675-a653-005c08734bd7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:10.618037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f1c30d21c1c4a67838e717b1ac5e1e0', 'request_id': 'req-9d1a8acd-45d5-4675-a653-005c08734bd7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:10.638302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f1c30d21c1c4a67838e717b1ac5e1e0', 'request_id': 'req-9d1a8acd-45d5-4675-a653-005c08734bd7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:10.675960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] The token cache is being invalidate because project 6f1c30d21c1c4a67838e717b1ac5e1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:10.676304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:10.680261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:10.690442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 695/1389] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:10 2022] DELETE /identity/v3/projects/6f1c30d21c1c4a67838e717b1ac5e1e0 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:10.691820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595777e-193d-4181-9764-15067ba8e36b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:10.692408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595777e-193d-4181-9764-15067ba8e36b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:10.693067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595777e-193d-4181-9764-15067ba8e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:10.736709 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b595777e-193d-4181-9764-15067ba8e36b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:10.738092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b595777e-193d-4181-9764-15067ba8e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:10.754638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:10.756748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:10.877687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 695/1390] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:10.886142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1dc57-e652-43c0-a9d5-676d25a737a8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:10.886508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1dc57-e652-43c0-a9d5-676d25a737a8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:10.887633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1dc57-e652-43c0-a9d5-676d25a737a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:10.913751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8acd-45d5-4675-a653-005c08734bd7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:10.922081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:10.922718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:10.922997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:10.923300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:10.927711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-33f1dc57-e652-43c0-a9d5-676d25a737a8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:10.928250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:10.929453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-33f1dc57-e652-43c0-a9d5-676d25a737a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:10.939257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 696/1391] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:10.940886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0879542-32a3-4d99-a017-4db4e90ee398 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:10.942234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0879542-32a3-4d99-a017-4db4e90ee398 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:10.942738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0879542-32a3-4d99-a017-4db4e90ee398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:10.944704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:10.946178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:10.965817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 696/1392] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:10.966496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:10.984399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d0879542-32a3-4d99-a017-4db4e90ee398 None None] MFA Rules not processed for user `8d144842b98e477aaef275278ad8db7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:10.985958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d0879542-32a3-4d99-a017-4db4e90ee398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:10.992395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f829a4-abfb-451e-9f17-e2120ad9331b service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:11.007722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:11.008406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.008609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.008878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.009523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:11.009824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:11.012667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:11.018108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:11.044241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:11.049855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 697/1393] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:11.050838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:11.072941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a7d3-1422-4ecf-a27b-fa2e34ba74df admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:11.099788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 697/1394] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:11.099788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.099788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.099788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.100283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e760e5-4eb3-4536-946b-e090f36cfc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:11.100283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e760e5-4eb3-4536-946b-e090f36cfc01 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.100283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e760e5-4eb3-4536-946b-e090f36cfc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.100283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e760e5-4eb3-4536-946b-e090f36cfc01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.100626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e760e5-4eb3-4536-946b-e090f36cfc01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:11.101450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e760e5-4eb3-4536-946b-e090f36cfc01 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:11.184775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] MFA Rules not processed for user `51b2f292cea24a21928e2adcef0c3786`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:11.186278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:11.192080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 698/1395] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:11 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:11.193380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0050b-a6d1-402a-9f44-587ad61d1739 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.193906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0050b-a6d1-402a-9f44-587ad61d1739 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.194232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0050b-a6d1-402a-9f44-587ad61d1739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.250903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-29c0050b-a6d1-402a-9f44-587ad61d1739 None None] MFA Rules not processed for user `8d144842b98e477aaef275278ad8db7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:11.255405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-29c0050b-a6d1-402a-9f44-587ad61d1739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:11.335699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 698/1396] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:11.338610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:11.345520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:11.394227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 699/1397] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:11.395321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0050b-a6d1-402a-9f44-587ad61d1739 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:11.505051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f09e49-20b6-4bea-8f5c-c65a9f6bd222 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:11.563089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0050b-a6d1-402a-9f44-587ad61d1739 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:11.575281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:11.576129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.576430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.578321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.580570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:11.585813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:11.608383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:11.609361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.609858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.610368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.612659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:11.643939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 700/1398] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:11 2022] POST /identity/v3/users => generated 291 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:11.644908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:11.646642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:11.685072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:11.765923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:11.767715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:11.846003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 699/1399] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:11.846003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:11.855579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2906e4d-d2ce-4add-81ac-74ff13fd0811 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:11.874315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6b4c1-a0d6-43d0-86ca-bfdb0a8a6135 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:11.895641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:11.904098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.904479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.904754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.905448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:11.907106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:11.911886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:11.912598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:11.913080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:11.913431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:11.919333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:11.947433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 700/1400] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:11.948448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:11.964477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:12.010816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644296e1-0df0-4a05-8fbc-4ea66a38675a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:12.021829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:12.022452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:12.022727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:12.022996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:12.042510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:12.044216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:12.054970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:12.057514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:12.069622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 701/1401] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:12.070462 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:12.071755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 701/1402] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:12.073431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:12.079822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:12.087322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1fc79-86af-4734-bfdc-3a5e030bddea service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:12.093803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52825f3-2419-4516-9225-e2afce0d1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:12.096173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52825f3-2419-4516-9225-e2afce0d1f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:12.096790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52825f3-2419-4516-9225-e2afce0d1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:12.097254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52825f3-2419-4516-9225-e2afce0d1f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:12.098615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52825f3-2419-4516-9225-e2afce0d1f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:12.100527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52825f3-2419-4516-9225-e2afce0d1f04 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:12.130446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 702/1403] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:12.132147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc4dd-d1ea-4203-b94c-7c99f775c509 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:12.135519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc4dd-d1ea-4203-b94c-7c99f775c509 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:12.136045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc4dd-d1ea-4203-b94c-7c99f775c509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:12.224067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b22cc4dd-d1ea-4203-b94c-7c99f775c509 None None] MFA Rules not processed for user `1e37d9ed3751483ebe4141706f46e43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:12.228347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b22cc4dd-d1ea-4203-b94c-7c99f775c509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:12.341648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d859c-eeb7-4d43-bd12-f13b83b5007f service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:12.349290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3c42a-c384-4ebb-80ae-85532fd65902 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:12.349920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3c42a-c384-4ebb-80ae-85532fd65902 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:12.350198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3c42a-c384-4ebb-80ae-85532fd65902 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:12.350476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3c42a-c384-4ebb-80ae-85532fd65902 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:12.352822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-18e3c42a-c384-4ebb-80ae-85532fd65902 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:12.371694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3c42a-c384-4ebb-80ae-85532fd65902 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:12.377466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3c42a-c384-4ebb-80ae-85532fd65902 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:12.423762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 703/1404] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:12.423762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cc4dd-d1ea-4203-b94c-7c99f775c509 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:12.441697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 702/1405] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:12.460714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cc4dd-d1ea-4203-b94c-7c99f775c509 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:12.484505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:12.489094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:12.489493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:12.494239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] PATH_INFO: `/v3/projects/399294fbeff94dd2a14c7f0318bd2550/users/5da640527cf645739da7a1a1a60f7e24/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:12.570469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399294fbeff94dd2a14c7f0318bd2550, user_id=5da640527cf645739da7a1a1a60f7e24, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:12.578274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:12.619411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 704/1406] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:12 2022] PUT /identity/v3/projects/399294fbeff94dd2a14c7f0318bd2550/users/5da640527cf645739da7a1a1a60f7e24/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:12.621900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615959-e96b-4f62-8be8-d574318456e6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:12.622187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615959-e96b-4f62-8be8-d574318456e6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:12.622456 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615959-e96b-4f62-8be8-d574318456e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:12.663461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dc615959-e96b-4f62-8be8-d574318456e6 None None] MFA Rules not processed for user `5da640527cf645739da7a1a1a60f7e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:12.664869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dc615959-e96b-4f62-8be8-d574318456e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:12.798772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 703/1407] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:12.799869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:12.854091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc615959-e96b-4f62-8be8-d574318456e6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:12.903675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:12.964841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96914879-4245-4b66-950f-61cb9fa07f45 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:12.979626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5630eac-2415-46b1-8bc6-e75ab09b5411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:12.980494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630eac-2415-46b1-8bc6-e75ab09b5411 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:12.981441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630eac-2415-46b1-8bc6-e75ab09b5411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:12.981813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630eac-2415-46b1-8bc6-e75ab09b5411 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:12.982448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5630eac-2415-46b1-8bc6-e75ab09b5411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:12.989747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5630eac-2415-46b1-8bc6-e75ab09b5411 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:12.997674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:13.009040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc615959-e96b-4f62-8be8-d574318456e6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:13.025907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:13.028793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:13.029236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:13.029517 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:13.057053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 705/1408] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:12 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:13.058928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23369df8-8640-4efd-a041-e51d07e9fea5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:13.059671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23369df8-8640-4efd-a041-e51d07e9fea5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:13.059921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23369df8-8640-4efd-a041-e51d07e9fea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:13.111810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-23369df8-8640-4efd-a041-e51d07e9fea5 None None] MFA Rules not processed for user `c987a5a52fc641f78054ac0218edcd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:13.117965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-23369df8-8640-4efd-a041-e51d07e9fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:13.222853 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:13.224511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:13.261392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 706/1409] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:13.261392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23369df8-8640-4efd-a041-e51d07e9fea5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:13.269620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 704/1410] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:13.430189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23369df8-8640-4efd-a041-e51d07e9fea5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:13.447732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:13.449062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:13.449345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:13.449771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:13.450496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:13.452535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:13.547871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:13.556147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 707/1411] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:13 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:13.562600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe53f2-4072-466c-bead-67c203ab1a98 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:13.586434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:13.587238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:13.596155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:13.596519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:13.598976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:13.602119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:13.673031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 705/1412] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:13.674154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:13.695255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8af86-2007-4efb-890d-91808603abac admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:13.718706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:13.719502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:13.719846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:13.720181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] PATH_INFO: `/v3/projects/fdd036bb40384b56bc39cd3ecc489b28/users/e33ce62b28fd4c8484a088a501855469/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:13.779882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd036bb40384b56bc39cd3ecc489b28, user_id=e33ce62b28fd4c8484a088a501855469, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:13.788121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:13.807084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 708/1413] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:13 2022] PUT /identity/v3/projects/fdd036bb40384b56bc39cd3ecc489b28/users/e33ce62b28fd4c8484a088a501855469/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:13.810895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:13.827496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f28b3-5b46-4214-b1b1-1554c0c7a589 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:13.968901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef8421-819a-4336-b77a-7b45ddfa63e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:13.969759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef8421-819a-4336-b77a-7b45ddfa63e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:13.970168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef8421-819a-4336-b77a-7b45ddfa63e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:13.970568 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef8421-819a-4336-b77a-7b45ddfa63e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:13.971384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef8421-819a-4336-b77a-7b45ddfa63e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:13.973128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef8421-819a-4336-b77a-7b45ddfa63e8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:13.992903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 706/1414] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:13.994965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:14.018480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4c9ff-d891-4ca3-8e19-fe36c7f83f29 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:14.029263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fcc5f-435e-44d8-88f5-b66ab57fb906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:14.040706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fcc5f-435e-44d8-88f5-b66ab57fb906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:14.041874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fcc5f-435e-44d8-88f5-b66ab57fb906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:14.042779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fcc5f-435e-44d8-88f5-b66ab57fb906 admin admin] PATH_INFO: `/v3/projects/fdd036bb40384b56bc39cd3ecc489b28/users/e33ce62b28fd4c8484a088a501855469/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:14.048288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fcc5f-435e-44d8-88f5-b66ab57fb906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd036bb40384b56bc39cd3ecc489b28, user_id=e33ce62b28fd4c8484a088a501855469, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:14.052161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fcc5f-435e-44d8-88f5-b66ab57fb906 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:14.108985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:14.109704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:14.110170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:14.111123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 709/1415] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:13 2022] PUT /identity/v3/projects/fdd036bb40384b56bc39cd3ecc489b28/users/e33ce62b28fd4c8484a088a501855469/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:14.185652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] MFA Rules not processed for user `e33ce62b28fd4c8484a088a501855469`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:14.187314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:14.278969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0a869-d409-41eb-9d31-ad318333ee08 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:14.279496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0a869-d409-41eb-9d31-ad318333ee08 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:14.279928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0a869-d409-41eb-9d31-ad318333ee08 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:14.281729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 710/1416] 10.222.0.36 () {54 vars in 912 bytes} [Mon Oct 17 07:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:03:14.288067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20027-36b2-4a4a-9e1b-fbc7f3d788dc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:14.288557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20027-36b2-4a4a-9e1b-fbc7f3d788dc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:14.289050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20027-36b2-4a4a-9e1b-fbc7f3d788dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:14.290397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:14.334865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b0c20027-36b2-4a4a-9e1b-fbc7f3d788dc None None] MFA Rules not processed for user `56967a5531ea4e308af5b89c13524c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:14.337919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b0c20027-36b2-4a4a-9e1b-fbc7f3d788dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:14.385388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 707/1417] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:14.399201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:14.403675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:14.453352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:14.484095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 711/1418] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:03:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:14.496392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c20027-36b2-4a4a-9e1b-fbc7f3d788dc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:14.522752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c20027-36b2-4a4a-9e1b-fbc7f3d788dc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:14.538821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:14.540372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:14.540953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:14.541749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:14.558550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:14.560082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:14.572319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013b9de-077c-4b63-bf52-74c61aad7b5c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:14.602790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecac57-146a-4987-bf57-c2ec6d24cd99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:14.603904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecac57-146a-4987-bf57-c2ec6d24cd99 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:14.604174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecac57-146a-4987-bf57-c2ec6d24cd99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:14.604443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecac57-146a-4987-bf57-c2ec6d24cd99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:14.606557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 712/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:14.636691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecac57-146a-4987-bf57-c2ec6d24cd99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:14.642683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecac57-146a-4987-bf57-c2ec6d24cd99 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:14.669541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 708/1420] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:14.983539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:14.997295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8679507-c539-4479-ab42-12ed5d3505f6 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:15.006859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00608d-59a1-406e-b4bb-084a9a8464bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:15.007583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00608d-59a1-406e-b4bb-084a9a8464bb service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:15.008839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00608d-59a1-406e-b4bb-084a9a8464bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:15.008839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00608d-59a1-406e-b4bb-084a9a8464bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:15.134466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00608d-59a1-406e-b4bb-084a9a8464bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:15.141258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00608d-59a1-406e-b4bb-084a9a8464bb service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:15.164779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 713/1421] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:16.262666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060de0-75ad-403d-a02b-1ac5685d5648 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:16.264108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060de0-75ad-403d-a02b-1ac5685d5648 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:16.264419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060de0-75ad-403d-a02b-1ac5685d5648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:16.289816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-97060de0-75ad-403d-a02b-1ac5685d5648 None None] MFA Rules not processed for user `51b2f292cea24a21928e2adcef0c3786`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:16.291066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-97060de0-75ad-403d-a02b-1ac5685d5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:16.376281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 709/1422] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:17.869194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:17.870234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:17.870742 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:17.914551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] MFA Rules not processed for user `1e37d9ed3751483ebe4141706f46e43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:17.916552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:18.088761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 714/1423] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:19.323672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2957e9-377e-4407-ba58-e32f3cf1a3d8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:19.326742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2957e9-377e-4407-ba58-e32f3cf1a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:19.327068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2957e9-377e-4407-ba58-e32f3cf1a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:19.584133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0c2957e9-377e-4407-ba58-e32f3cf1a3d8 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:19.593822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0c2957e9-377e-4407-ba58-e32f3cf1a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:19.755515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:19.800139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 710/1424] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:03:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:20.246050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:20.248511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:20.529733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:20.551877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8693beb-f779-4fd8-8436-3f0b6362f75c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:20.582572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92e3a3-0f97-4637-923a-b751229cf438 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:20.584156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92e3a3-0f97-4637-923a-b751229cf438 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:20.584368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92e3a3-0f97-4637-923a-b751229cf438 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:20.584653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92e3a3-0f97-4637-923a-b751229cf438 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:20.598593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92e3a3-0f97-4637-923a-b751229cf438 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:20.598593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92e3a3-0f97-4637-923a-b751229cf438 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:20.599855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2957e9-377e-4407-ba58-e32f3cf1a3d8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:20.609046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 715/1425] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:03:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:03:20.717580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2957e9-377e-4407-ba58-e32f3cf1a3d8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:20.729588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:20.730238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:20.730581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:20.730903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] PATH_INFO: `/v3/users/5da640527cf645739da7a1a1a60f7e24` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:20.739444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da640527cf645739da7a1a1a60f7e24)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:20.741431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:20.966909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da640527cf645739da7a1a1a60f7e24'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:20.967981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da640527cf645739da7a1a1a60f7e24'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.034753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da640527cf645739da7a1a1a60f7e24'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.053600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da640527cf645739da7a1a1a60f7e24'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.054405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da640527cf645739da7a1a1a60f7e24'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.131788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 711/1426] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:20 2022] DELETE /identity/v3/users/5da640527cf645739da7a1a1a60f7e24 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:21.137604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879721-da62-430e-b689-478d4e032236 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:21.137875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879721-da62-430e-b689-478d4e032236 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:21.138145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879721-da62-430e-b689-478d4e032236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:21.192503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-39879721-da62-430e-b689-478d4e032236 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:21.193843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-39879721-da62-430e-b689-478d4e032236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:21.378272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 716/1427] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:21.401735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:21.404267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:21.539424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879721-da62-430e-b689-478d4e032236 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:21.544832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-39879721-da62-430e-b689-478d4e032236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:21.561157 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff279f8c-69e9-4f69-9e31-880727bf3639 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:21.573025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c1914-e54b-4bcc-9725-09c134e09fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:21.573917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1914-e54b-4bcc-9725-09c134e09fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:21.574196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1914-e54b-4bcc-9725-09c134e09fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:21.574534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1914-e54b-4bcc-9725-09c134e09fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:21.577375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879721-da62-430e-b689-478d4e032236 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:21.588134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c1914-e54b-4bcc-9725-09c134e09fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:21.590641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c1914-e54b-4bcc-9725-09c134e09fa6 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:21.599449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:21.601700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:21.605095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:21.605637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] PATH_INFO: `/v3/users/0863a3df7ae840608827c1865c3a784c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:21.608530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0863a3df7ae840608827c1865c3a784c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:21.620528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:21.624566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 712/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:21.737174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0863a3df7ae840608827c1865c3a784c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.737869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0863a3df7ae840608827c1865c3a784c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.745902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0863a3df7ae840608827c1865c3a784c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.747157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0863a3df7ae840608827c1865c3a784c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.790156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0863a3df7ae840608827c1865c3a784c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:21.838928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 717/1429] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:21 2022] DELETE /identity/v3/users/0863a3df7ae840608827c1865c3a784c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:21.843901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:21.844142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:21.844739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:21.895061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:21.898054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:21.928705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:22.049388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 713/1430] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:22.064828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:22.073548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:22.175063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed55f-13f4-4407-a70f-301560c826c0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:22.181739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:22.182468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:22.182642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:22.183394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:22.199392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:22.201443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:22.215697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 718/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:22.984087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:22.985722 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:23.009846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:23.011412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:23.028627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeded5-e7af-4aab-996e-4f5185fb96e9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:23.034975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:23.035827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:23.036265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:23.036678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] PATH_INFO: `/v3/projects/63468386eaae4369b44f37a4235217c3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:23.038010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=63468386eaae4369b44f37a4235217c3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:23.041014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f039d4-2dd8-4b53-a0b0-4e971ca97b8a service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:23.041759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:23.054942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:23.055690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:23.055969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:23.056301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] PATH_INFO: `/v3/projects/399294fbeff94dd2a14c7f0318bd2550` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:23.059861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399294fbeff94dd2a14c7f0318bd2550)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:23.061798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:23.065121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:23.158242 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399294fbeff94dd2a14c7f0318bd2550', 'request_id': 'req-8b68b0d0-f764-45fc-bd3e-22c48faaee97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.167951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399294fbeff94dd2a14c7f0318bd2550', 'request_id': 'req-8b68b0d0-f764-45fc-bd3e-22c48faaee97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.175307 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399294fbeff94dd2a14c7f0318bd2550', 'request_id': 'req-8b68b0d0-f764-45fc-bd3e-22c48faaee97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.177893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63468386eaae4369b44f37a4235217c3', 'request_id': 'req-a449d6de-704d-4a69-960d-cec173caf7af'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.191716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63468386eaae4369b44f37a4235217c3', 'request_id': 'req-a449d6de-704d-4a69-960d-cec173caf7af'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.199128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] The token cache is being invalidate because project 399294fbeff94dd2a14c7f0318bd2550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:23.200576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.203167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.206898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 719/1432] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:23 2022] DELETE /identity/v3/projects/399294fbeff94dd2a14c7f0318bd2550 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:23.213375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:23.215663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:23.220590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63468386eaae4369b44f37a4235217c3', 'request_id': 'req-a449d6de-704d-4a69-960d-cec173caf7af'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.251918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] The token cache is being invalidate because project 63468386eaae4369b44f37a4235217c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:23.253475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.257206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.260124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 714/1433] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:22 2022] DELETE /identity/v3/projects/63468386eaae4369b44f37a4235217c3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:23.266728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:23.268562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:23.357627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68b0d0-f764-45fc-bd3e-22c48faaee97 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:23.364928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:23.366415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:23.366804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:23.367186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] PATH_INFO: `/v3/users/e33ce62b28fd4c8484a088a501855469` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:23.368706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33ce62b28fd4c8484a088a501855469)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:23.371493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:23.435902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449d6de-704d-4a69-960d-cec173caf7af admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:23.450545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:23.452351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:23.453150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:23.453874 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] PATH_INFO: `/v3/users/0177790fdf0f4feeb415aadbfe4d45f6` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:23.455898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0177790fdf0f4feeb415aadbfe4d45f6)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:23.456624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33ce62b28fd4c8484a088a501855469'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.457967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33ce62b28fd4c8484a088a501855469'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.461874 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:23.477012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33ce62b28fd4c8484a088a501855469'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.480169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33ce62b28fd4c8484a088a501855469'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.499966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2fd8cc7-42bf-4243-9af4-ac7e72043207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33ce62b28fd4c8484a088a501855469'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.529708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 720/1434] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:23 2022] DELETE /identity/v3/users/e33ce62b28fd4c8484a088a501855469 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:23.578031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:23.614844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0177790fdf0f4feeb415aadbfe4d45f6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.615689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0177790fdf0f4feeb415aadbfe4d45f6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.648090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0177790fdf0f4feeb415aadbfe4d45f6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.680075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0177790fdf0f4feeb415aadbfe4d45f6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.681664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0177790fdf0f4feeb415aadbfe4d45f6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:23.774542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 715/1435] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:23 2022] DELETE /identity/v3/users/0177790fdf0f4feeb415aadbfe4d45f6 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:24.342640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:24.344036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:24.344603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:24.408019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:24.410409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:24.431492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] MFA Rules not processed for user `56967a5531ea4e308af5b89c13524c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:24.442582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:24.489383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:24.752164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 721/1436] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 17 07:03:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:24.789400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbe8bc-cbb6-4fa1-84db-4f3de7214f79 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:24.800666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:24.801441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:24.801655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:24.801934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] PATH_INFO: `/v3/projects/fdd036bb40384b56bc39cd3ecc489b28` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:24.803964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd036bb40384b56bc39cd3ecc489b28)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:24.809908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:24.832682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:24.834336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:24.854760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee41a-48ba-469f-a2e2-a4f1dbb8546f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:24.864430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:24.865120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:24.867446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:24.867812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:24.885705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:24.888329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:24.914198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 722/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:24.929292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:24.941996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0e548-7fb1-4523-a16c-0d36137e759e service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:24.952327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5becc1-b354-4d05-a889-1f0e66205d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:24.952951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5becc1-b354-4d05-a889-1f0e66205d38 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:24.953345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5becc1-b354-4d05-a889-1f0e66205d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:24.953795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5becc1-b354-4d05-a889-1f0e66205d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:24.968679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5becc1-b354-4d05-a889-1f0e66205d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:24.972023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5becc1-b354-4d05-a889-1f0e66205d38 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:24.984479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 723/1438] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:25.015277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd036bb40384b56bc39cd3ecc489b28', 'request_id': 'req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:25.027158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd036bb40384b56bc39cd3ecc489b28', 'request_id': 'req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:25.053914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd036bb40384b56bc39cd3ecc489b28', 'request_id': 'req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:25.084616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] The token cache is being invalidate because project fdd036bb40384b56bc39cd3ecc489b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:25.085278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:25.086167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f07d7bbb-bd5b-4063-bdf0-9a9c15d16526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:25.088970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 716/1439] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:24 2022] DELETE /identity/v3/projects/fdd036bb40384b56bc39cd3ecc489b28 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:25.093843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39f6d-b385-4907-b406-27e94511093f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:25.094516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39f6d-b385-4907-b406-27e94511093f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:25.095297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39f6d-b385-4907-b406-27e94511093f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:25.158769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-72a39f6d-b385-4907-b406-27e94511093f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:25.161488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-72a39f6d-b385-4907-b406-27e94511093f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:25.456929 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 724/1440] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:25.459968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:25.460478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:25.461775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:25.528541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:25.530750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:25.552147 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a39f6d-b385-4907-b406-27e94511093f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:25.575900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 717/1441] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:25.589415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a39f6d-b385-4907-b406-27e94511093f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:25.612569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:25.617205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:25.617484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:25.617849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:25.621022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:25.621378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:25.623100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:25.689528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:25.692264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:25.697981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 725/1442] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:25.703873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:25.724335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52950196-4675-4ef0-8f42-fa51d1a01890 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:25.742734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:25.743820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:25.744215 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:25.744535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] PATH_INFO: `/v3/projects/067c5a629ef344d8ba723b9c88433640` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:25.746625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] RBAC: Authorizing `identity:delete_project(project_id=067c5a629ef344d8ba723b9c88433640)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:25.748717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:25.754767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274cf72-4922-45cb-a2c6-e79e7265fe24 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:25.791949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:25.794661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:25.795195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:25.795709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:25.796765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:25.814651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:25.918130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 726/1443] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:25 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:25.922494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:25.931242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:25.962661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067c5a629ef344d8ba723b9c88433640', 'request_id': 'req-136c853e-ee0e-4506-b017-d9c39bc1bdda'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:25.984146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067c5a629ef344d8ba723b9c88433640', 'request_id': 'req-136c853e-ee0e-4506-b017-d9c39bc1bdda'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:26.005290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067c5a629ef344d8ba723b9c88433640', 'request_id': 'req-136c853e-ee0e-4506-b017-d9c39bc1bdda'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:26.072898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] The token cache is being invalidate because project 067c5a629ef344d8ba723b9c88433640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:26.073973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:26.075653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-136c853e-ee0e-4506-b017-d9c39bc1bdda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:26.081392 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 718/1444] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:25 2022] DELETE /identity/v3/projects/067c5a629ef344d8ba723b9c88433640 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:26.083301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769ebe-e971-4c60-bf94-a339309fb902 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.083757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769ebe-e971-4c60-bf94-a339309fb902 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.084192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769ebe-e971-4c60-bf94-a339309fb902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.130527 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-92769ebe-e971-4c60-bf94-a339309fb902 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:26.131896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-92769ebe-e971-4c60-bf94-a339309fb902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:26.142374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:26.171702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf529a7-d7d7-4b0c-845b-5df6a6c5b507 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.332640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.333454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.333859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.334250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.335724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.339163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:26.355162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 719/1445] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:26.357491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e41d6-a251-49fc-8202-80b22b148059 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.357777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e41d6-a251-49fc-8202-80b22b148059 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.358054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e41d6-a251-49fc-8202-80b22b148059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.412149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 727/1446] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:25 2022] POST /identity/v3/users => generated 289 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:26.414059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d19e41d6-a251-49fc-8202-80b22b148059 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:26.416554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:26.419795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d19e41d6-a251-49fc-8202-80b22b148059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:26.424091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:26.447394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 720/1447] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:26.448725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e41d6-a251-49fc-8202-80b22b148059 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:26.464190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f25406-ab95-4c4e-a623-4d1db1d0b492 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.466026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e41d6-a251-49fc-8202-80b22b148059 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.474695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.476269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.476269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.476269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.476775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:26.477082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.477639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.478506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.479006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.479115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:26.481338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.482244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.484951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:26.506866 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 721/1448] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:26.509392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 728/1449] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:26.511062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:26.517244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:26.529988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d98f6-1219-48c8-9028-971929779bb4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.535628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289ea3c-cb3e-4700-9166-dc7881392ed2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.540932 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.541561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.541843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.542117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.542812 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.548266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:26.551454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.552100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.552806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.552806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.553322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.555098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:26.577461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 729/1450] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:26.580660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:26.599640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3880ab-c3a8-4cc5-a949-39088cb4c1e1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.609202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 722/1451] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:26 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:26.610775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc9ad-94a5-440e-94b6-5546c2fc37dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.611391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc9ad-94a5-440e-94b6-5546c2fc37dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.612374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:26.613034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc9ad-94a5-440e-94b6-5546c2fc37dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.613319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc9ad-94a5-440e-94b6-5546c2fc37dc admin admin] PATH_INFO: `/v3/projects/02c694f808c14642b58b1d82137f8c88/users/ac5b857f64e44a2f99a4fb9b100e18ca/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.652884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dc9ad-94a5-440e-94b6-5546c2fc37dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c694f808c14642b58b1d82137f8c88, user_id=ac5b857f64e44a2f99a4fb9b100e18ca, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.655735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dc9ad-94a5-440e-94b6-5546c2fc37dc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:26.687537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 730/1452] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:26 2022] PUT /identity/v3/projects/02c694f808c14642b58b1d82137f8c88/users/ac5b857f64e44a2f99a4fb9b100e18ca/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:26.703545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce841b-52b6-427a-8b40-6c54d2df4760 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.704169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce841b-52b6-427a-8b40-6c54d2df4760 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.704452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce841b-52b6-427a-8b40-6c54d2df4760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.783723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4802a2-0a82-4b9d-a69e-83cb660650a0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.786171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-06ce841b-52b6-427a-8b40-6c54d2df4760 None None] MFA Rules not processed for user `ac5b857f64e44a2f99a4fb9b100e18ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:26.788148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-06ce841b-52b6-427a-8b40-6c54d2df4760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:26.851504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:26.938799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 731/1453] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:26.943372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce841b-52b6-427a-8b40-6c54d2df4760 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:26.944097 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.945170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.945606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.946024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.946806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.949253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:26.963308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce841b-52b6-427a-8b40-6c54d2df4760 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:26.973148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:26.975880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:26.976332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:26.976770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:26.977833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:26.984127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.002435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 723/1454] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:26 2022] POST /identity/v3/users => generated 292 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:27.003411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.024171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0c98d-cf50-406e-a850-45d072e5c404 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.037029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.037740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.038028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.038302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.040179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.048296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.049578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 732/1455] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:26 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:27.051406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.076200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 724/1456] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:27.077387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.082026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:27.102405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f75cf4-f0dd-4097-abc1-f56ef9a5d95a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.183585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfed3b3-4637-4b8f-85a0-43c936db3e9b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.201004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.201722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.202094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.202473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.203194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.205979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.223479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.224240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.224519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.224786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.225481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.228121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.240801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 733/1457] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:27 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:27.241007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 725/1458] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:27.242057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.247429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.257304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44af2e8-9e57-47b3-a054-c4a2899ff80f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.261365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e32cc1-f12d-401a-a9db-084bb11e0c95 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.264704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.265771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.266215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.266826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.267275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378614e0-a957-4a80-843f-e3cd6e631ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.267880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.268122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378614e0-a957-4a80-843f-e3cd6e631ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.268402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378614e0-a957-4a80-843f-e3cd6e631ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.268751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378614e0-a957-4a80-843f-e3cd6e631ad7 admin admin] PATH_INFO: `/v3/projects/d4af98cb71ae4c28b9771edbbb386300/users/ad583f8722d04a25860c913ed69fc077/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.275685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.287075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378614e0-a957-4a80-843f-e3cd6e631ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4af98cb71ae4c28b9771edbbb386300, user_id=ad583f8722d04a25860c913ed69fc077, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.292647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378614e0-a957-4a80-843f-e3cd6e631ad7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.294110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 734/1459] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:27.295676 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.306341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 726/1460] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:27 2022] PUT /identity/v3/projects/d4af98cb71ae4c28b9771edbbb386300/users/ad583f8722d04a25860c913ed69fc077/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:27.306617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294ba63-9575-480b-a232-162fdd86ac4d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.317900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6518b5-5be2-4ecf-aeab-adca22d26f0b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.318221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6518b5-5be2-4ecf-aeab-adca22d26f0b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.318533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6518b5-5be2-4ecf-aeab-adca22d26f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.353038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4a6518b5-5be2-4ecf-aeab-adca22d26f0b None None] MFA Rules not processed for user `ad583f8722d04a25860c913ed69fc077`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:27.354493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4a6518b5-5be2-4ecf-aeab-adca22d26f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:27.399078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.399814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.400285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.400609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.401283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.403058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.415798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 735/1461] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:27.421324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.434734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1389d-3320-4563-bfc1-7d3c7ad54b33 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.439754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 727/1462] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:27.441270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6518b5-5be2-4ecf-aeab-adca22d26f0b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.444231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bfe86-1a25-4e41-a684-ca39e5479ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.445060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bfe86-1a25-4e41-a684-ca39e5479ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.445481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bfe86-1a25-4e41-a684-ca39e5479ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.446047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bfe86-1a25-4e41-a684-ca39e5479ea3 admin admin] PATH_INFO: `/v3/projects/81d16edeb58a424aae9bcbb60c75244d/users/1c9d6912590f4ae5831c782e8f019c62/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.453364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6518b5-5be2-4ecf-aeab-adca22d26f0b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.458871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.459975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.460916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.461290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.462036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.464301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.465506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bfe86-1a25-4e41-a684-ca39e5479ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d16edeb58a424aae9bcbb60c75244d, user_id=1c9d6912590f4ae5831c782e8f019c62, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.468360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bfe86-1a25-4e41-a684-ca39e5479ea3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.470578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:27.490237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 736/1463] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:27 2022] PUT /identity/v3/projects/81d16edeb58a424aae9bcbb60c75244d/users/1c9d6912590f4ae5831c782e8f019c62/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:27.501959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991150c8-9722-491f-8962-e00a354d9a58 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.502287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991150c8-9722-491f-8962-e00a354d9a58 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.502630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991150c8-9722-491f-8962-e00a354d9a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.510882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 728/1464] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:27 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:27.515172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.548843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-991150c8-9722-491f-8962-e00a354d9a58 None None] MFA Rules not processed for user `1c9d6912590f4ae5831c782e8f019c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:27.550280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-991150c8-9722-491f-8962-e00a354d9a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:27.609074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7672aa-1567-40a0-99f9-7a4337811bfc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.615532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.616729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.617687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.617687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.618347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.620388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.628731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:27.642566 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 737/1465] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:27.648341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991150c8-9722-491f-8962-e00a354d9a58 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.649498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 729/1466] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:27 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:27.653565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.663818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e726d-df79-46d8-bbe0-bfec6cffbf1b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.665073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991150c8-9722-491f-8962-e00a354d9a58 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.668991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.669599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.669873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.670300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.670941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.671830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.672566 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.672668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.672996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.673363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.674197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.676533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.685526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 730/1467] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:03:27.688627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.701137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010ecc4-7861-40ec-af64-4c6617459a20 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.711004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.711707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.711985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.712244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] PATH_INFO: `/v3/projects/85274a7024bf4e12977f362db650165c/users/bfac1fc85957440eb5f59596e1f0bb05/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.721179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 738/1468] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:27 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:27.721934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.731344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85274a7024bf4e12977f362db650165c, user_id=bfac1fc85957440eb5f59596e1f0bb05, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.734066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.749112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 731/1469] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:27 2022] PUT /identity/v3/projects/85274a7024bf4e12977f362db650165c/users/bfac1fc85957440eb5f59596e1f0bb05/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:27.752981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.854004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa39e2-dd44-4e1c-baf2-2829cfe299e6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.863302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec954-b715-489a-8e54-1924488999c3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.872292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.874057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.875662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.876551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.879162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.883452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.885050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.887182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.887665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.888634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.890798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.894255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.923250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 732/1470] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:03:27.933854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.934041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 739/1471] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:27 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:27.940693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:27.958946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161d4f-85cd-4589-ab6a-de231b6c492e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.962778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8367d09-5b5c-491c-b14c-99e373a7a0da admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:27.964774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.967083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.967471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.967953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] PATH_INFO: `/v3/projects/85274a7024bf4e12977f362db650165c/users/bfac1fc85957440eb5f59596e1f0bb05/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.971256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85274a7024bf4e12977f362db650165c, user_id=bfac1fc85957440eb5f59596e1f0bb05, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.975265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:27.978824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab638-e283-4871-8126-741850cf753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:27.979544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab638-e283-4871-8126-741850cf753d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:27.979825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab638-e283-4871-8126-741850cf753d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:27.980161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab638-e283-4871-8126-741850cf753d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:27.981610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ab638-e283-4871-8126-741850cf753d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:27.983510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ab638-e283-4871-8126-741850cf753d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:28.002897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 733/1472] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:27 2022] PUT /identity/v3/projects/85274a7024bf4e12977f362db650165c/users/bfac1fc85957440eb5f59596e1f0bb05/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:28.003092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 740/1473] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:28.003183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:28.004837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba32f5-8c8a-4d45-bb4f-f68c4ea0a994 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.005131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba32f5-8c8a-4d45-bb4f-f68c4ea0a994 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.005411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba32f5-8c8a-4d45-bb4f-f68c4ea0a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.059887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-23ba32f5-8c8a-4d45-bb4f-f68c4ea0a994 None None] MFA Rules not processed for user `bfac1fc85957440eb5f59596e1f0bb05`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:28.061391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-23ba32f5-8c8a-4d45-bb4f-f68c4ea0a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:28.072668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:28.156929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e1db-ec1e-498e-a3ed-42d06a3b26c2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:28.169561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d00c9-3e3a-40b4-bf49-6f08afd0671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:28.170127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d00c9-3e3a-40b4-bf49-6f08afd0671a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.170450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d00c9-3e3a-40b4-bf49-6f08afd0671a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.170717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d00c9-3e3a-40b4-bf49-6f08afd0671a admin admin] PATH_INFO: `/v3/projects/cca6434214304a148e1fb0868527b0f5/users/815efe2c36b148429366b965d583f74d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.197453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d00c9-3e3a-40b4-bf49-6f08afd0671a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca6434214304a148e1fb0868527b0f5, user_id=815efe2c36b148429366b965d583f74d, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:28.198999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 741/1474] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:28.201240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92678f77-f640-46f0-80de-25d3e300e150 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.201619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92678f77-f640-46f0-80de-25d3e300e150 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.201986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92678f77-f640-46f0-80de-25d3e300e150 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.204403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d00c9-3e3a-40b4-bf49-6f08afd0671a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:28.211339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 742/1475] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:03:28.212689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92678f77-f640-46f0-80de-25d3e300e150 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:28.216124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-92678f77-f640-46f0-80de-25d3e300e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:28.229463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 734/1476] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:28 2022] PUT /identity/v3/projects/cca6434214304a148e1fb0868527b0f5/users/815efe2c36b148429366b965d583f74d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:28.229463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e374f-374b-45fc-9429-63bc964cb31e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.229463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e374f-374b-45fc-9429-63bc964cb31e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.229463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e374f-374b-45fc-9429-63bc964cb31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.255657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-747e374f-374b-45fc-9429-63bc964cb31e None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:28.257113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-747e374f-374b-45fc-9429-63bc964cb31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:28.289383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:28.319666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92678f77-f640-46f0-80de-25d3e300e150 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:28.324895 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:28.325684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.326130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.326542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.350200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 735/1477] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:03:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:28.351083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e374f-374b-45fc-9429-63bc964cb31e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:28.414609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:28.415807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:28.427090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 743/1478] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:28.428043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:28.429819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:28.441764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e374f-374b-45fc-9429-63bc964cb31e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:28.447057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:28.447875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.448168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.448769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.449624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:28.451314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:28.457360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675764b-5aa6-4456-a4e2-4dc874c36478 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:28.462755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 736/1479] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:28.465270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9864e-9c32-4b2e-bc47-f72fd729c8cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:28.466103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9864e-9c32-4b2e-bc47-f72fd729c8cd service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.467445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:28.467957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9864e-9c32-4b2e-bc47-f72fd729c8cd service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.468432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9864e-9c32-4b2e-bc47-f72fd729c8cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.477480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215e99-d7bd-4f8f-a733-0b094c397390 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:28.478736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9864e-9c32-4b2e-bc47-f72fd729c8cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:28.481268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9864e-9c32-4b2e-bc47-f72fd729c8cd service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:28.482910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:28.483636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.483921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.484194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] PATH_INFO: `/v3/projects/cca6434214304a148e1fb0868527b0f5/users/815efe2c36b148429366b965d583f74d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.486135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca6434214304a148e1fb0868527b0f5, user_id=815efe2c36b148429366b965d583f74d, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:28.488897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:28.495507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 744/1480] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:28.507248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 737/1481] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:28 2022] PUT /identity/v3/projects/cca6434214304a148e1fb0868527b0f5/users/815efe2c36b148429366b965d583f74d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:28.515583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db3c3-5935-434c-bfbe-07285036906e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:28.515931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db3c3-5935-434c-bfbe-07285036906e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:28.516287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db3c3-5935-434c-bfbe-07285036906e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:28.555556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-285db3c3-5935-434c-bfbe-07285036906e None None] MFA Rules not processed for user `815efe2c36b148429366b965d583f74d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:28.557477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-285db3c3-5935-434c-bfbe-07285036906e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:28.695986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 745/1482] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:28.875852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:28.877647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:28.937403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:29.063442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c40c21-f694-4c35-beca-fffd06f58d2e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:29.077686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:29.078679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:29.079125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:29.079739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:29.095195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:29.095661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:29.114142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 738/1483] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:29.315769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db3c3-5935-434c-bfbe-07285036906e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:29.347966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db3c3-5935-434c-bfbe-07285036906e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:29.358294 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:29.359303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:29.359739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:29.360249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:29.478317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:29.481253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:29.493721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 746/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:35.745396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:35.760002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356d7df-6b7b-4a2d-8fd7-1c223fe63bdb service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:35.767067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0699c-40c2-4ee1-95fb-721010864ee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:35.768167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0699c-40c2-4ee1-95fb-721010864ee7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:35.768666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0699c-40c2-4ee1-95fb-721010864ee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:35.769318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0699c-40c2-4ee1-95fb-721010864ee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:35.901765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0699c-40c2-4ee1-95fb-721010864ee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:35.903882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0699c-40c2-4ee1-95fb-721010864ee7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:35.918297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 739/1485] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:36.066516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:36.070790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:36.200225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa79f4-409e-4142-998b-5f441160388e service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:36.205551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:36.206213 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:36.206491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:36.206764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] PATH_INFO: `/v3/users/ac5b857f64e44a2f99a4fb9b100e18ca` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:36.207998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5b857f64e44a2f99a4fb9b100e18ca)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:36.210132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:36.283291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5b857f64e44a2f99a4fb9b100e18ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:36.284439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5b857f64e44a2f99a4fb9b100e18ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:36.294989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5b857f64e44a2f99a4fb9b100e18ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:36.296112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5b857f64e44a2f99a4fb9b100e18ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:36.314645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5b857f64e44a2f99a4fb9b100e18ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:36.335750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 747/1486] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:36 2022] DELETE /identity/v3/users/ac5b857f64e44a2f99a4fb9b100e18ca => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:36.340156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ba18f-350b-43c3-8b1c-2fb9717463de None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:36.340665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ba18f-350b-43c3-8b1c-2fb9717463de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:36.340792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ba18f-350b-43c3-8b1c-2fb9717463de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:36.500278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9a0ba18f-350b-43c3-8b1c-2fb9717463de None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:36.501646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9a0ba18f-350b-43c3-8b1c-2fb9717463de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:36.596860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 740/1487] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:36.624343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:36.626417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:36.700911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68ee4d-056d-40ca-abcb-20515588a65a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:36.705024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37621443-b284-40b5-a5a2-04641defbd7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:36.705712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621443-b284-40b5-a5a2-04641defbd7c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:36.706096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621443-b284-40b5-a5a2-04641defbd7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:36.706625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621443-b284-40b5-a5a2-04641defbd7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:36.717412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37621443-b284-40b5-a5a2-04641defbd7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:36.719186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37621443-b284-40b5-a5a2-04641defbd7c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:36.730803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 748/1488] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:37.843626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97ec8-83be-450d-9074-2a1dca766c81 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:37.848185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97ec8-83be-450d-9074-2a1dca766c81 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:37.848846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97ec8-83be-450d-9074-2a1dca766c81 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:37.851453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 741/1489] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:03:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:03:37.907343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:37.908683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:37.909443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:37.953380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:37.957003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:38.014376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:38.085291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 749/1490] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:03:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:38.283297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97ec8-83be-450d-9074-2a1dca766c81 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:38.285239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-25e97ec8-83be-450d-9074-2a1dca766c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:38.304944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97ec8-83be-450d-9074-2a1dca766c81 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:38.313380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:38.314441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:38.314873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:38.315656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:38.340547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:38.344734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:38.385964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 742/1491] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:38.506287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:38.507829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:38.525543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f6b0e-43f1-4d1b-b1fe-47f0ae3eda5f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:38.531476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:38.532545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:38.532797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:38.533142 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] PATH_INFO: `/v3/projects/02c694f808c14642b58b1d82137f8c88` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:38.536107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c694f808c14642b58b1d82137f8c88)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:38.536491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:38.619960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c694f808c14642b58b1d82137f8c88', 'request_id': 'req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.632213 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c694f808c14642b58b1d82137f8c88', 'request_id': 'req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.649678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c694f808c14642b58b1d82137f8c88', 'request_id': 'req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.690345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] The token cache is being invalidate because project 02c694f808c14642b58b1d82137f8c88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:38.693178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.695622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.697898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 750/1492] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:38 2022] DELETE /identity/v3/projects/02c694f808c14642b58b1d82137f8c88 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:38.702674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:38.717161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409f85-3cde-42bb-9a85-7ed181a397f4 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:38.804873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:38.826537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:38.827673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:38.828101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:38.829425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] PATH_INFO: `/v3/users/1c9d6912590f4ae5831c782e8f019c62` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:38.830641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c9d6912590f4ae5831c782e8f019c62)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:38.835545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:38.937309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c9d6912590f4ae5831c782e8f019c62'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.939744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c9d6912590f4ae5831c782e8f019c62'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.950869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c9d6912590f4ae5831c782e8f019c62'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.962521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c9d6912590f4ae5831c782e8f019c62'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.964047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c9d6912590f4ae5831c782e8f019c62'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:38.991320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 743/1493] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:38 2022] DELETE /identity/v3/users/1c9d6912590f4ae5831c782e8f019c62 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:39.413840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:39.415384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:39.446231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e6d8d-0df3-4f10-8106-70e6c5cbd250 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:39.545965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:39.546862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:39.547149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:39.547463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] PATH_INFO: `/v3/projects/81d16edeb58a424aae9bcbb60c75244d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:39.548839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d16edeb58a424aae9bcbb60c75244d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:39.550801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:39.626279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:39.634162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d16edeb58a424aae9bcbb60c75244d', 'request_id': 'req-eb48f5fc-8d75-4c23-9078-283bea5246f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.641430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d16edeb58a424aae9bcbb60c75244d', 'request_id': 'req-eb48f5fc-8d75-4c23-9078-283bea5246f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.649084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d16edeb58a424aae9bcbb60c75244d', 'request_id': 'req-eb48f5fc-8d75-4c23-9078-283bea5246f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.664851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] The token cache is being invalidate because project 81d16edeb58a424aae9bcbb60c75244d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:39.665260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.665912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.671100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 751/1494] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:39 2022] DELETE /identity/v3/projects/81d16edeb58a424aae9bcbb60c75244d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:39.672402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:39.673836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:39.689259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f3bcf-44f2-4891-a280-d756ceb0b5e9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:39.883477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:39.884466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:39.884746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:39.885158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] PATH_INFO: `/v3/users/815efe2c36b148429366b965d583f74d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:39.891556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815efe2c36b148429366b965d583f74d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:39.893723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:39.972299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815efe2c36b148429366b965d583f74d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.973715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815efe2c36b148429366b965d583f74d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.982417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815efe2c36b148429366b965d583f74d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.991329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815efe2c36b148429366b965d583f74d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.992107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-835f07ef-18c0-4796-aaa4-00770ae1a673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815efe2c36b148429366b965d583f74d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:39.994860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:40.022828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 744/1495] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:39 2022] DELETE /identity/v3/users/815efe2c36b148429366b965d583f74d => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:40.077947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:40.079598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:40.097111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48f5fc-8d75-4c23-9078-283bea5246f8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:40.164744 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:40.165383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.165660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.165930 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] PATH_INFO: `/v3/users/65a69676a2bf48cbbee53f6c112e37bc` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.167418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a69676a2bf48cbbee53f6c112e37bc)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:40.170383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:40.231428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a69676a2bf48cbbee53f6c112e37bc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.232429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a69676a2bf48cbbee53f6c112e37bc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.242437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a69676a2bf48cbbee53f6c112e37bc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.243607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a69676a2bf48cbbee53f6c112e37bc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.298785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a69676a2bf48cbbee53f6c112e37bc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.319509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 752/1496] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:40 2022] DELETE /identity/v3/users/65a69676a2bf48cbbee53f6c112e37bc => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:40.324765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.325173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.325551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.348900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:40.350894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:40.431259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 745/1497] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:40.443079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:40.444702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:40.504923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:40.510309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652f31-a7c5-4212-b18f-587be1d21070 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:40.519837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd09600-c2ca-4d4a-9a19-0f9421128afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:40.524019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd09600-c2ca-4d4a-9a19-0f9421128afe service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.524277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd09600-c2ca-4d4a-9a19-0f9421128afe service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.525763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd09600-c2ca-4d4a-9a19-0f9421128afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.538039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd09600-c2ca-4d4a-9a19-0f9421128afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:40.539903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd09600-c2ca-4d4a-9a19-0f9421128afe service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:40.546496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:40.548136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:40.562297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b35b1-525b-4715-9264-fd8faef25d83 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:40.567332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:40.568102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 753/1498] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:40.568254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.568444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.568768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] PATH_INFO: `/v3/projects/cca6434214304a148e1fb0868527b0f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.569929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca6434214304a148e1fb0868527b0f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:40.571976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:40.632810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca6434214304a148e1fb0868527b0f5', 'request_id': 'req-b96daa39-82f1-4a82-a86f-b8c563753dc9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.642874 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca6434214304a148e1fb0868527b0f5', 'request_id': 'req-b96daa39-82f1-4a82-a86f-b8c563753dc9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.654464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca6434214304a148e1fb0868527b0f5', 'request_id': 'req-b96daa39-82f1-4a82-a86f-b8c563753dc9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.670199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] The token cache is being invalidate because project cca6434214304a148e1fb0868527b0f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:40.671056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.671926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b96daa39-82f1-4a82-a86f-b8c563753dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:40.674385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 746/1499] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:40 2022] DELETE /identity/v3/projects/cca6434214304a148e1fb0868527b0f5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:40.680895 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce5f11-ca2d-4649-a227-78bc0a6dc7e8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.681183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce5f11-ca2d-4649-a227-78bc0a6dc7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.681459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce5f11-ca2d-4649-a227-78bc0a6dc7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.703716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-23ce5f11-ca2d-4649-a227-78bc0a6dc7e8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:40.705191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-23ce5f11-ca2d-4649-a227-78bc0a6dc7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:40.768767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 754/1500] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:40.776792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b47bc-914e-4dd4-bf3f-78daefc3283e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.777652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b47bc-914e-4dd4-bf3f-78daefc3283e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.777962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b47bc-914e-4dd4-bf3f-78daefc3283e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.798696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-293b47bc-914e-4dd4-bf3f-78daefc3283e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:40.799936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-293b47bc-914e-4dd4-bf3f-78daefc3283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:40.807890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 747/1501] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:40.812296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce5f11-ca2d-4649-a227-78bc0a6dc7e8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:40.822069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce5f11-ca2d-4649-a227-78bc0a6dc7e8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:40.827658 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:40.828253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.828532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.828810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.829452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:40.829755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:40.831395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:40.844467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 755/1502] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:40.848960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b47bc-914e-4dd4-bf3f-78daefc3283e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:40.859202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b47bc-914e-4dd4-bf3f-78daefc3283e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:40.864641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:40.865347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.865631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.865902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.866532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:40.868494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:40.903202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 748/1503] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:40 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:40.904564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:40.916401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc975c-4f64-4a37-a8d0-c6433eb9077a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:40.991819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:40.993839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:40.996107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:40.996842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:40.997202 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:40.997583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:40.998278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.000485 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.012292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffd2f7-6083-4437-b390-4477c632b3aa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.018648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.019457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.020093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.020590 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] PATH_INFO: `/v3/projects/629158ad423c4e6c90af6bf991fa4831` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.024028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629158ad423c4e6c90af6bf991fa4831)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.026780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.032557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 756/1504] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:40 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:41.036204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.046112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf3b02-3457-4884-97a8-ec4d10d21fe8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.054107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.055318 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.055606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.056377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.057030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.059629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.071776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:41.072270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 757/1505] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:03:41.076676 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.088370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a02cc-f423-4e83-a4cf-4cc9934515ba admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.090270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629158ad423c4e6c90af6bf991fa4831', 'request_id': 'req-0761aa49-044b-490f-8831-8f8e49e24bc2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.096683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629158ad423c4e6c90af6bf991fa4831', 'request_id': 'req-0761aa49-044b-490f-8831-8f8e49e24bc2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.105002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629158ad423c4e6c90af6bf991fa4831', 'request_id': 'req-0761aa49-044b-490f-8831-8f8e49e24bc2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.116736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] The token cache is being invalidate because project 629158ad423c4e6c90af6bf991fa4831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:41.117024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.117713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.119303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 749/1506] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:40 2022] DELETE /identity/v3/projects/629158ad423c4e6c90af6bf991fa4831 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:41.123744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.125090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:41.144252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:41.160116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.160725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.161001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.161300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.162332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.163725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.179399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 758/1507] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:41.181291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.183803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:41.196944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d29409-b83d-4339-a4a0-901b01b4aed8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.202433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a360f-0d1a-4c26-8bae-abc60581a098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.203062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a360f-0d1a-4c26-8bae-abc60581a098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.203375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a360f-0d1a-4c26-8bae-abc60581a098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.203644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a360f-0d1a-4c26-8bae-abc60581a098 admin admin] PATH_INFO: `/v3/projects/a56fba95abb442f49586e0f259af0c3d/users/ea4c5a8950124d60bcb553cd959709c9/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.204992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761aa49-044b-490f-8831-8f8e49e24bc2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.210794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.211492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.211773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.214542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] PATH_INFO: `/v3/users/f38c62f644af4853bdad65cd464c9b69` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.215326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38c62f644af4853bdad65cd464c9b69)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.217303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.219809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a360f-0d1a-4c26-8bae-abc60581a098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56fba95abb442f49586e0f259af0c3d, user_id=ea4c5a8950124d60bcb553cd959709c9, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.222502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a360f-0d1a-4c26-8bae-abc60581a098 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.251696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 759/1508] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:41 2022] PUT /identity/v3/projects/a56fba95abb442f49586e0f259af0c3d/users/ea4c5a8950124d60bcb553cd959709c9/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:41.266151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86076007-b5a0-47ab-badf-09cec4a661f6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.266560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86076007-b5a0-47ab-badf-09cec4a661f6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.267221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86076007-b5a0-47ab-badf-09cec4a661f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.284550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38c62f644af4853bdad65cd464c9b69'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.285838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38c62f644af4853bdad65cd464c9b69'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.297985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38c62f644af4853bdad65cd464c9b69'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.304295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-86076007-b5a0-47ab-badf-09cec4a661f6 None None] MFA Rules not processed for user `ea4c5a8950124d60bcb553cd959709c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:41.305606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-86076007-b5a0-47ab-badf-09cec4a661f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:41.307744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38c62f644af4853bdad65cd464c9b69'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.308516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38c62f644af4853bdad65cd464c9b69'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.340113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 750/1509] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:41 2022] DELETE /identity/v3/users/f38c62f644af4853bdad65cd464c9b69 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:41.396297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 760/1510] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:41.400964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.402480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:41.450572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:41.478086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58785-4ed9-4bac-a32f-d363b3e8375a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.483502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.484108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.484381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.484646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.485424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.487344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.517705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 751/1511] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:41 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:41.522457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86076007-b5a0-47ab-badf-09cec4a661f6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.589786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86076007-b5a0-47ab-badf-09cec4a661f6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.661725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.662336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.662610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.662880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.663553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.666084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.677569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:41.702029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 761/1512] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:41 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:41.706149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.708047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.709754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:41.718090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe3d3-ab2f-4317-9dba-43c5322a49b5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.723831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.724496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.724724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.724994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.725559 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd1947-1170-4a7f-9e3c-5a6ddac1ac6a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.725713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.727555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.735864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.736577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.739089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.739404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] PATH_INFO: `/v3/projects/2bef9510338246e899acbf71e4a57adf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.739968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 752/1513] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:03:41.743005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bef9510338246e899acbf71e4a57adf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.744338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.745008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.754474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa016045-f54b-4a29-838f-11228c72d432 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.759871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.761113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.761437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.761707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] PATH_INFO: `/v3/projects/2bf30161647b41b6923e18cd8583422a/users/4f20f94f3f174fed91d8bb0890a37a7b/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.776356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf30161647b41b6923e18cd8583422a, user_id=4f20f94f3f174fed91d8bb0890a37a7b, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.779865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.797567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bef9510338246e899acbf71e4a57adf', 'request_id': 'req-476dcb52-b813-4f5c-b5ab-2d1277540004'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.803974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bef9510338246e899acbf71e4a57adf', 'request_id': 'req-476dcb52-b813-4f5c-b5ab-2d1277540004'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.810624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 753/1514] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:41 2022] PUT /identity/v3/projects/2bf30161647b41b6923e18cd8583422a/users/4f20f94f3f174fed91d8bb0890a37a7b/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:41.814562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bef9510338246e899acbf71e4a57adf', 'request_id': 'req-476dcb52-b813-4f5c-b5ab-2d1277540004'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.814712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.829855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] The token cache is being invalidate because project 2bef9510338246e899acbf71e4a57adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:41.830136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.830918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-476dcb52-b813-4f5c-b5ab-2d1277540004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:41.832780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 762/1515] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:41 2022] DELETE /identity/v3/projects/2bef9510338246e899acbf71e4a57adf => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:41.837197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a586df-55a0-401e-bfbe-9f2e3d3b2f30 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.837480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a586df-55a0-401e-bfbe-9f2e3d3b2f30 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.837755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a586df-55a0-401e-bfbe-9f2e3d3b2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.861980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-78a586df-55a0-401e-bfbe-9f2e3d3b2f30 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:41.863380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-78a586df-55a0-401e-bfbe-9f2e3d3b2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:41.888042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795248b-c2d6-42ba-806f-9f6c904d355a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.893227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.894091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.894466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.894669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.895371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.896988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.909623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 754/1516] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:03:41.913963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:41.916542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:41.930260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b45c6-35cc-4516-ab6f-1ba7f67b9136 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:41.935647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff446a4-5d0f-4566-96cb-d9fb74d4ab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:41.936305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff446a4-5d0f-4566-96cb-d9fb74d4ab4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.936719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff446a4-5d0f-4566-96cb-d9fb74d4ab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.936999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff446a4-5d0f-4566-96cb-d9fb74d4ab4b admin admin] PATH_INFO: `/v3/projects/2bf30161647b41b6923e18cd8583422a/users/4f20f94f3f174fed91d8bb0890a37a7b/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.937853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 763/1517] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:41.939011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff446a4-5d0f-4566-96cb-d9fb74d4ab4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf30161647b41b6923e18cd8583422a, user_id=4f20f94f3f174fed91d8bb0890a37a7b, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:41.941733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff446a4-5d0f-4566-96cb-d9fb74d4ab4b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:41.947557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01599b-fb54-436a-8ed4-a15139dd70a2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.948118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01599b-fb54-436a-8ed4-a15139dd70a2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.948118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01599b-fb54-436a-8ed4-a15139dd70a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.954458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 755/1518] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:41 2022] PUT /identity/v3/projects/2bf30161647b41b6923e18cd8583422a/users/4f20f94f3f174fed91d8bb0890a37a7b/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:41.964531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:41.964900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:41.965246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:41.970601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ac01599b-fb54-436a-8ed4-a15139dd70a2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:41.971858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ac01599b-fb54-436a-8ed4-a15139dd70a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:42.002830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] MFA Rules not processed for user `4f20f94f3f174fed91d8bb0890a37a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:42.004188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:42.039006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 764/1519] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:42.043657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01599b-fb54-436a-8ed4-a15139dd70a2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.052270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01599b-fb54-436a-8ed4-a15139dd70a2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.056856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.057435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.057713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.057994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.058634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:42.058934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.060542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.067696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:42.071983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 765/1520] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:03:42.074483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 756/1521] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:42.077384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.086117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.087042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d321437-1507-47da-b7aa-b1b1125722c5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.087476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:42.091771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.092556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.092836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.093107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.094233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.096137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.100703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:42.127947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 766/1522] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:42 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:42.131989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.161513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d3d5e-eb0c-4a9b-b9e3-28a2a6432168 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.208433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ff8d0-e733-4855-8627-0da8a69f1a69 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.213238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.213875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.214193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.214801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.215143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.217494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.228390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.229001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.229491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.229605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.247424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 767/1523] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:42 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:42.248934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.262152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c19c9-c80e-4fe7-99f5-33502d93f0e7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.267915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.268523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.268802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.269073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.269708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.271622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.287023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 768/1524] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:03:42.288027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.300155 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f7319-907e-4108-87dc-762642449b99 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.301335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.302778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.304814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d1856-010a-443b-ac9c-2bb7fae95ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.305395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1856-010a-443b-ac9c-2bb7fae95ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.305748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1856-010a-443b-ac9c-2bb7fae95ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.306021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1856-010a-443b-ac9c-2bb7fae95ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.306689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1856-010a-443b-ac9c-2bb7fae95ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.308454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1856-010a-443b-ac9c-2bb7fae95ff5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.312364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 757/1525] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:42.319511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 769/1526] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:42.323459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.334834 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f038-81e2-4fb4-a090-34a4e7e0d5ff service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.340410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979116c-690b-432c-a169-006feac82494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.341051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979116c-690b-432c-a169-006feac82494 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.342372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979116c-690b-432c-a169-006feac82494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.342651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979116c-690b-432c-a169-006feac82494 admin admin] PATH_INFO: `/v3/projects/bb7f3b3962c844dbaea2be84531ae14d/users/61663e807ff140e4b6f06daf86e0ced7/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.360191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979116c-690b-432c-a169-006feac82494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7f3b3962c844dbaea2be84531ae14d, user_id=61663e807ff140e4b6f06daf86e0ced7, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.363239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979116c-690b-432c-a169-006feac82494 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.378388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 758/1527] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:42 2022] PUT /identity/v3/projects/bb7f3b3962c844dbaea2be84531ae14d/users/61663e807ff140e4b6f06daf86e0ced7/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:42.397769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231eb15-a04f-4bf0-81ef-7deaccdebaed None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.397769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231eb15-a04f-4bf0-81ef-7deaccdebaed None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.397769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231eb15-a04f-4bf0-81ef-7deaccdebaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.406327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979116c-690b-432c-a169-006feac82494 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.408035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d979116c-690b-432c-a169-006feac82494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:42.433753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7231eb15-a04f-4bf0-81ef-7deaccdebaed None None] MFA Rules not processed for user `61663e807ff140e4b6f06daf86e0ced7`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:42.435120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7231eb15-a04f-4bf0-81ef-7deaccdebaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:42.475620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:42.493488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979116c-690b-432c-a169-006feac82494 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.498518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.499538 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.499822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.500100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.502120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:42.507402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 770/1528] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:42.509629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231eb15-a04f-4bf0-81ef-7deaccdebaed None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.561884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:42.570736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.572241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.581292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 759/1529] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:42.582162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231eb15-a04f-4bf0-81ef-7deaccdebaed None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.586736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.587372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.587652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.587923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.588542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.590939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.623419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 771/1530] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:42 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:42.627122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.637557 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f25a3e-c415-4041-b47f-75ed0f33adc0 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.704613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.705231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.705509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.705781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.706408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.708425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.736435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 760/1531] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:42 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:42.740689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.751812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb185bff-aa5f-46e1-8c98-8bce8bc90983 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.757118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.757727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.758006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.759444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.760364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.768979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.786461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 772/1532] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:42.788163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.801511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d21bb-53d3-4914-b89b-0f5db96a56a2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.806179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.806867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.807145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.807458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] PATH_INFO: `/v3/projects/4f8aca0a3fa5454597bd3f2f4796db78/users/5b0a0b2ae9564194bc981d911577aa9a/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.823890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8aca0a3fa5454597bd3f2f4796db78, user_id=5b0a0b2ae9564194bc981d911577aa9a, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.827105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.842551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 761/1533] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:42 2022] PUT /identity/v3/projects/4f8aca0a3fa5454597bd3f2f4796db78/users/5b0a0b2ae9564194bc981d911577aa9a/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:42.846485 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.857931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecfe59-4d4d-4d1c-b197-0141bd1ea079 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.876340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:42.961673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92a7d1-cb42-4167-9002-91eeadfc0e15 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:42.966725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:42.967412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:42.967690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:42.968071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] PATH_INFO: `/v3/users/ea4c5a8950124d60bcb553cd959709c9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:42.969345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4c5a8950124d60bcb553cd959709c9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:42.971364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:42.973395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:43.044675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4c5a8950124d60bcb553cd959709c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.045437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4c5a8950124d60bcb553cd959709c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.055248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4c5a8950124d60bcb553cd959709c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.056577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.057622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.058023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.058465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.059294 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:43.061297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:43.069157 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4c5a8950124d60bcb553cd959709c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.071042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abf3780b-8a54-482b-acef-69ab8b010d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4c5a8950124d60bcb553cd959709c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.075574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 773/1534] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:43.079164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:43.092538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.113478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee207f6f-fdf1-4970-98c6-01983f3a3353 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:43.120301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc8f84-e0c7-4205-86a7-3be822dc91fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.122010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc8f84-e0c7-4205-86a7-3be822dc91fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.122661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc8f84-e0c7-4205-86a7-3be822dc91fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.123116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc8f84-e0c7-4205-86a7-3be822dc91fa admin admin] PATH_INFO: `/v3/projects/4f8aca0a3fa5454597bd3f2f4796db78/users/5b0a0b2ae9564194bc981d911577aa9a/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.128609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 762/1535] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:42 2022] DELETE /identity/v3/users/ea4c5a8950124d60bcb553cd959709c9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:43.128773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc8f84-e0c7-4205-86a7-3be822dc91fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8aca0a3fa5454597bd3f2f4796db78, user_id=5b0a0b2ae9564194bc981d911577aa9a, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:43.130151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.130439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.130716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.134250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc8f84-e0c7-4205-86a7-3be822dc91fa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:43.150901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 774/1536] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:43 2022] PUT /identity/v3/projects/4f8aca0a3fa5454597bd3f2f4796db78/users/5b0a0b2ae9564194bc981d911577aa9a/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:43.161469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.161469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.161853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.172961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:43.174545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.201763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] MFA Rules not processed for user `5b0a0b2ae9564194bc981d911577aa9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:43.203943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.273150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 763/1537] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:43.284030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:43.286618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.317760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 775/1538] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:43.331536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:43.334218 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.376289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:43.386755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd6095-dd3b-4fc8-8c73-e9a0c0d213ee None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:43.396174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.397646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.398159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.398632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.411610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:43.413433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:43.414828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a75204-9553-4066-a9de-9fed75125ec2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:43.423198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.424840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.424955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.425192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.426109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 764/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:43.437983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:43.439691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:43.452626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 776/1540] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:43.497556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:43.499603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.514184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a647b-8a22-4b19-b4b3-83c0476f1511 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:43.520319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.521085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.521732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.522421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] PATH_INFO: `/v3/users/ad583f8722d04a25860c913ed69fc077` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.524111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad583f8722d04a25860c913ed69fc077)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:43.526252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:43.540593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:43.549295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:43.552487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f69e3-6863-4b8f-b804-c3d7724e4629 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:43.578606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.579244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.579557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.579881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] PATH_INFO: `/v3/users/61663e807ff140e4b6f06daf86e0ced7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.582011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61663e807ff140e4b6f06daf86e0ced7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:43.584625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:43.615708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad583f8722d04a25860c913ed69fc077'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.616959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad583f8722d04a25860c913ed69fc077'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.628084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad583f8722d04a25860c913ed69fc077'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.648562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad583f8722d04a25860c913ed69fc077'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.649600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f08fccf-36fe-4941-b40b-6ce6ee9dfe12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad583f8722d04a25860c913ed69fc077'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.664334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61663e807ff140e4b6f06daf86e0ced7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.666682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61663e807ff140e4b6f06daf86e0ced7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.682427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61663e807ff140e4b6f06daf86e0ced7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.683559 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61663e807ff140e4b6f06daf86e0ced7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.692793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 765/1541] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:43 2022] DELETE /identity/v3/users/ad583f8722d04a25860c913ed69fc077 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:43.697776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.698063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.698344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.724089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:43.725087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43ed2bd4-1be4-4936-aaf7-bc30363acac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61663e807ff140e4b6f06daf86e0ced7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:43.725335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.746809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 777/1542] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:43 2022] DELETE /identity/v3/users/61663e807ff140e4b6f06daf86e0ced7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:43.750894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.751789 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.751789 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.791779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:43.793776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.829100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 766/1543] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:43.837520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:43.839119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.878739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 778/1544] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:43.885908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:43.888409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:43.925841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8649805-8973-473c-b581-4bae2c81f338 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:43.949627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:43.968061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050597e8-0788-4beb-abc5-31fa650a9652 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:43.972254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.972963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.973388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.973726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:43.986228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:43.987926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:43.995507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:43.996725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:43.997006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:43.997334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:44.000793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 779/1545] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:44.006967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:44.012282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:44.014502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:44.028436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 767/1546] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:44.035525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:44.037177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:44.056460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695634-29d1-4f97-9849-337b9341d333 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:44.062503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:44.063344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:44.063761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:44.063911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] PATH_INFO: `/v3/projects/a56fba95abb442f49586e0f259af0c3d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:44.065093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56fba95abb442f49586e0f259af0c3d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:44.067041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:44.123591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56fba95abb442f49586e0f259af0c3d', 'request_id': 'req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.129846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56fba95abb442f49586e0f259af0c3d', 'request_id': 'req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.136840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56fba95abb442f49586e0f259af0c3d', 'request_id': 'req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.153226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] The token cache is being invalidate because project a56fba95abb442f49586e0f259af0c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:44.153565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.154333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.159360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 780/1547] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:44 2022] DELETE /identity/v3/projects/a56fba95abb442f49586e0f259af0c3d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:44.161386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:44.173941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3be7-5f98-403f-bb3f-1bd8c5a838cb service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:44.236033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:44.236665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:44.236942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:44.237216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] PATH_INFO: `/v3/users/4f20f94f3f174fed91d8bb0890a37a7b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:44.238537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f20f94f3f174fed91d8bb0890a37a7b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:44.240523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:44.290137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f20f94f3f174fed91d8bb0890a37a7b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.290808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f20f94f3f174fed91d8bb0890a37a7b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.298610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f20f94f3f174fed91d8bb0890a37a7b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.306338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f20f94f3f174fed91d8bb0890a37a7b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.306955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f20f94f3f174fed91d8bb0890a37a7b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.340227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 768/1548] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:44 2022] DELETE /identity/v3/users/4f20f94f3f174fed91d8bb0890a37a7b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:44.577627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:44.579185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:44.592186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:44.597923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:44.610354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c260b-b9a0-46c4-866e-c4fa5aa8cb4c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:44.700706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:44.703411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:44.703948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:44.704243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] PATH_INFO: `/v3/projects/d4af98cb71ae4c28b9771edbbb386300` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:44.707364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4af98cb71ae4c28b9771edbbb386300)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:44.709568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:44.730436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4adec-ad96-478c-b17a-534c295d819c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:44.735557 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:44.736195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:44.736475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:44.736885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] PATH_INFO: `/v3/projects/bb7f3b3962c844dbaea2be84531ae14d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:44.738026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb7f3b3962c844dbaea2be84531ae14d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:44.739985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:44.758999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:44.798893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4af98cb71ae4c28b9771edbbb386300', 'request_id': 'req-6b1ded7e-c1d3-4b71-81da-db59beec6ade'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.804603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4af98cb71ae4c28b9771edbbb386300', 'request_id': 'req-6b1ded7e-c1d3-4b71-81da-db59beec6ade'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.812277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4af98cb71ae4c28b9771edbbb386300', 'request_id': 'req-6b1ded7e-c1d3-4b71-81da-db59beec6ade'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.817245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:44.822724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb7f3b3962c844dbaea2be84531ae14d', 'request_id': 'req-7f3ae1ca-c518-4f13-8440-15240946f620'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.830559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb7f3b3962c844dbaea2be84531ae14d', 'request_id': 'req-7f3ae1ca-c518-4f13-8440-15240946f620'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.831150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] The token cache is being invalidate because project d4af98cb71ae4c28b9771edbbb386300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:44.831635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.833456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.835477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 781/1549] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:44 2022] DELETE /identity/v3/projects/d4af98cb71ae4c28b9771edbbb386300 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:44.836968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:44.839264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb7f3b3962c844dbaea2be84531ae14d', 'request_id': 'req-7f3ae1ca-c518-4f13-8440-15240946f620'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.841529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:44.852790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] The token cache is being invalidate because project bb7f3b3962c844dbaea2be84531ae14d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:44.853186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.853840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.856016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 769/1550] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:44 2022] DELETE /identity/v3/projects/bb7f3b3962c844dbaea2be84531ae14d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:44.857025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:44.858860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:44.923413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ded7e-c1d3-4b71-81da-db59beec6ade admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:44.928333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:44.928992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:44.929274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:44.929548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] PATH_INFO: `/v3/projects/2bf30161647b41b6923e18cd8583422a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:44.930661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf30161647b41b6923e18cd8583422a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:44.931015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ae1ca-c518-4f13-8440-15240946f620 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:44.932604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:44.936407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:44.937216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:44.937497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:44.937763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] PATH_INFO: `/v3/users/bfac1fc85957440eb5f59596e1f0bb05` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:44.938883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfac1fc85957440eb5f59596e1f0bb05)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:44.940999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:44.988800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf30161647b41b6923e18cd8583422a', 'request_id': 'req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.991733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfac1fc85957440eb5f59596e1f0bb05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.992424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfac1fc85957440eb5f59596e1f0bb05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:44.994471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf30161647b41b6923e18cd8583422a', 'request_id': 'req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.000193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfac1fc85957440eb5f59596e1f0bb05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.001222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf30161647b41b6923e18cd8583422a', 'request_id': 'req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.007406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfac1fc85957440eb5f59596e1f0bb05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.008104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2d0edf85-cf7e-4c76-9c77-1d1d0b69a4c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfac1fc85957440eb5f59596e1f0bb05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.014541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] The token cache is being invalidate because project 2bf30161647b41b6923e18cd8583422a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:45.014904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.015790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.017716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 782/1551] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:44 2022] DELETE /identity/v3/projects/2bf30161647b41b6923e18cd8583422a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:45.021201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.022072 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.026183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 770/1552] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:44 2022] DELETE /identity/v3/users/bfac1fc85957440eb5f59596e1f0bb05 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:45.028133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaea77-9b35-42b2-8d02-e061c97ddb94 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.028430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaea77-9b35-42b2-8d02-e061c97ddb94 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.028689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaea77-9b35-42b2-8d02-e061c97ddb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.049227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-adeaea77-9b35-42b2-8d02-e061c97ddb94 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:45.051072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-adeaea77-9b35-42b2-8d02-e061c97ddb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.073919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:45.099562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebe3e5-b908-4ff3-b252-c0b0e94ffbdc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.134745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 771/1553] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:45.144787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db82ec-3b7e-4e3c-b99a-6d14ec84260d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.145271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db82ec-3b7e-4e3c-b99a-6d14ec84260d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.146214 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db82ec-3b7e-4e3c-b99a-6d14ec84260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.165715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:45.185553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-39db82ec-3b7e-4e3c-b99a-6d14ec84260d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:45.187098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-39db82ec-3b7e-4e3c-b99a-6d14ec84260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.190108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.192604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.192889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.193161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] PATH_INFO: `/v3/users/5b0a0b2ae9564194bc981d911577aa9a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.194319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0a0b2ae9564194bc981d911577aa9a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.195712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 772/1554] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:03:45.196414 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.200875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db82ec-3b7e-4e3c-b99a-6d14ec84260d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.212388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db82ec-3b7e-4e3c-b99a-6d14ec84260d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.218196 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.218995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.219461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.219801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.220652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:45.221022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.222773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.239960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 773/1555] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:45.248211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.262532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc46b0-3547-40dc-9934-af3174f14bdb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.267640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.268247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.268528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.268808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.269428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.271549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.273575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0a0b2ae9564194bc981d911577aa9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.274683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0a0b2ae9564194bc981d911577aa9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.285422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0a0b2ae9564194bc981d911577aa9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.288893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0a0b2ae9564194bc981d911577aa9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.305444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 774/1556] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:45.309457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.309706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0a0b2ae9564194bc981d911577aa9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.311259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.328873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 783/1557] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:45 2022] DELETE /identity/v3/users/5b0a0b2ae9564194bc981d911577aa9a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:45.396876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0a1f7-6808-49a0-b99e-98ecf6948658 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.413330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.414938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.484062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.484734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.485043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.485322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.486047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295926-5fe7-4d91-8453-3319a2198b1b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.486166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.488204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.490813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.491477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.491751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.492017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] PATH_INFO: `/v3/projects/85274a7024bf4e12977f362db650165c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.493313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85274a7024bf4e12977f362db650165c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.495234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.513789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 775/1558] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:45.517782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.528633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd6f2e-b29c-4660-9a5d-896ad7ed18e5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.537661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.538367 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.538653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.539020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.539713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.541418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.553178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 776/1559] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:45.557063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.562404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85274a7024bf4e12977f362db650165c', 'request_id': 'req-aecfa094-2aba-469c-84d0-4b067982f466'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.567630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:45.569904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376459-4789-4f16-95f8-146e9c2372d1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.570444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85274a7024bf4e12977f362db650165c', 'request_id': 'req-aecfa094-2aba-469c-84d0-4b067982f466'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.575680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:45.580586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85274a7024bf4e12977f362db650165c', 'request_id': 'req-aecfa094-2aba-469c-84d0-4b067982f466'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.596988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] The token cache is being invalidate because project 85274a7024bf4e12977f362db650165c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:45.597275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.598054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aecfa094-2aba-469c-84d0-4b067982f466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.599637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 784/1560] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:45 2022] DELETE /identity/v3/projects/85274a7024bf4e12977f362db650165c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:45.605406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54500ff-4c74-4e2f-8ef9-0b263d988a76 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.605678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54500ff-4c74-4e2f-8ef9-0b263d988a76 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.606364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54500ff-4c74-4e2f-8ef9-0b263d988a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.627891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d54500ff-4c74-4e2f-8ef9-0b263d988a76 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:45.629099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d54500ff-4c74-4e2f-8ef9-0b263d988a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.647948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.648866 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.649275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.649550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.650185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.651809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.663084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 777/1561] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:03:45.668516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.670565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.686633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06940-9ce3-4d55-b66a-ae1195b878da admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.693019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.693493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.693772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.694103 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] PATH_INFO: `/v3/projects/4293d6ff2cf548ae821148f9db4ba235/users/b115e8fb0d9d45e8a9ffe52c79fea777/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.704795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 785/1562] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:45.713024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433d719-179e-4aa7-bbed-8da86ec5bfdb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.713451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433d719-179e-4aa7-bbed-8da86ec5bfdb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.713632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4293d6ff2cf548ae821148f9db4ba235, user_id=b115e8fb0d9d45e8a9ffe52c79fea777, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.713813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433d719-179e-4aa7-bbed-8da86ec5bfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.716997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.736822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 778/1563] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:45 2022] PUT /identity/v3/projects/4293d6ff2cf548ae821148f9db4ba235/users/b115e8fb0d9d45e8a9ffe52c79fea777/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:45.737905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.740626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5433d719-179e-4aa7-bbed-8da86ec5bfdb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:45.741387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.741945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5433d719-179e-4aa7-bbed-8da86ec5bfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.825758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1374f-5662-45f0-a9c9-307b043f01ab admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:45.831498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:45.832210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.832490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.832763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] PATH_INFO: `/v3/projects/4f8aca0a3fa5454597bd3f2f4796db78` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.833902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f8aca0a3fa5454597bd3f2f4796db78)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:45.835852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:45.838836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 786/1564] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:45.840534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7ad3-182e-4036-a468-824e97e234fa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.843481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7ad3-182e-4036-a468-824e97e234fa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.844387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7ad3-182e-4036-a468-824e97e234fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.881419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e6ca7ad3-182e-4036-a468-824e97e234fa None None] MFA Rules not processed for user `b115e8fb0d9d45e8a9ffe52c79fea777`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:45.882723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e6ca7ad3-182e-4036-a468-824e97e234fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.898763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f8aca0a3fa5454597bd3f2f4796db78', 'request_id': 'req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.905738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f8aca0a3fa5454597bd3f2f4796db78', 'request_id': 'req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.916395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f8aca0a3fa5454597bd3f2f4796db78', 'request_id': 'req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.930594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] The token cache is being invalidate because project 4f8aca0a3fa5454597bd3f2f4796db78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:45.930789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.931493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:45.933323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 779/1565] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:45 2022] DELETE /identity/v3/projects/4f8aca0a3fa5454597bd3f2f4796db78 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:45.933857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:45.935183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.947164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:45.960961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 787/1566] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:45.962233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0f5d-4fa5-46c6-a562-6cdd3beeadfa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:45.962517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0f5d-4fa5-46c6-a562-6cdd3beeadfa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:45.963146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0f5d-4fa5-46c6-a562-6cdd3beeadfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:45.989158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b30b0f5d-4fa5-46c6-a562-6cdd3beeadfa None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:45.990373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b30b0f5d-4fa5-46c6-a562-6cdd3beeadfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:45.999921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:46.027860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911154e-a1ca-40a2-9f93-8fdaa6cb86e0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.033036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.033642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.033915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.034262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.035053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:46.035393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.037186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.054337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 780/1567] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:46.055268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.056913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.061469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 788/1568] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:46.062131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0f5d-4fa5-46c6-a562-6cdd3beeadfa None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.072530 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0f5d-4fa5-46c6-a562-6cdd3beeadfa None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.077666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078ab69-abda-4e73-9ea3-988ecf34905d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.078268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078ab69-abda-4e73-9ea3-988ecf34905d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.078547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078ab69-abda-4e73-9ea3-988ecf34905d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.078823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078ab69-abda-4e73-9ea3-988ecf34905d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.079476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078ab69-abda-4e73-9ea3-988ecf34905d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.081409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078ab69-abda-4e73-9ea3-988ecf34905d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.113915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 789/1569] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:46.115691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469c41-c083-46b5-b30d-770e6b8a4846 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.115971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469c41-c083-46b5-b30d-770e6b8a4846 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.116243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469c41-c083-46b5-b30d-770e6b8a4846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.128038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172bbe0-0caa-4a82-8734-c04e3f93e2d6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.138824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fc469c41-c083-46b5-b30d-770e6b8a4846 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:46.140350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fc469c41-c083-46b5-b30d-770e6b8a4846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.200969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.201603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.201794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.202082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.218335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 790/1570] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:46.219486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc469c41-c083-46b5-b30d-770e6b8a4846 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.230116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc469c41-c083-46b5-b30d-770e6b8a4846 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.237610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.237610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.237610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.237610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.237996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.239825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.264269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 791/1571] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/users => generated 286 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:46.265595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.266763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.268246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.275458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8221c-4a8d-4b07-9376-bd92393be302 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.277378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 781/1572] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:46.278215 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.280778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.281490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.281773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.282043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.282680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:46.282979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.285382 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.292501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854289de-0663-44d9-b599-bb9bdbc0a835 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.297711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.298319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.299770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.299770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.299969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 792/1573] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:46.300155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.301701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.303859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.312989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 782/1574] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:46.313897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.314809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b2f66-2949-4fe1-8fd8-463859ddc177 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.315830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.319419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.320002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.320278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.320546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.321167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.323048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.323436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:46.330296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c53f6c-9805-4b5d-af5f-3b9efc4ad650 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.334858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.335497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.335772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.336047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] PATH_INFO: `/v3/users/b115e8fb0d9d45e8a9ffe52c79fea777` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.337187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b115e8fb0d9d45e8a9ffe52c79fea777)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.339087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.350625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 793/1575] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:46.351604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.392926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b115e8fb0d9d45e8a9ffe52c79fea777'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:46.393593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b115e8fb0d9d45e8a9ffe52c79fea777'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:46.395767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:46.403168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b115e8fb0d9d45e8a9ffe52c79fea777'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:46.411313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b115e8fb0d9d45e8a9ffe52c79fea777'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:46.411964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b115e8fb0d9d45e8a9ffe52c79fea777'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:46.418890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b12909-cf94-4871-b455-97d42aa5744a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.425698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3e5e7-e02d-4faa-92a0-6f4143a97fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.426477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3e5e7-e02d-4faa-92a0-6f4143a97fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.426603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3e5e7-e02d-4faa-92a0-6f4143a97fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.426839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3e5e7-e02d-4faa-92a0-6f4143a97fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.427497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3e5e7-e02d-4faa-92a0-6f4143a97fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.429081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3e5e7-e02d-4faa-92a0-6f4143a97fe7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.431588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 783/1576] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:46 2022] DELETE /identity/v3/users/b115e8fb0d9d45e8a9ffe52c79fea777 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:46.435320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.435320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.445461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 794/1577] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:46.445461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.445461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.445461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.464209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:46.465523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.515693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e5486-669b-45f0-afd3-efbf89098bc7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.520717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.521332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.521613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.521884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.523659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.526059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.541090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 795/1578] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:46.543153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.544820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.559868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388393a3-a68d-40d7-a538-52f3c930b55c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.560068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 784/1579] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/users => generated 305 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:46.562079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.564849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.566766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.567547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.567926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.568354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] PATH_INFO: `/v3/projects/2dd523b1ee3849c98fb3625b9d1f9715/users/24304216b9ca483cbd21d493ae552929/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.610287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd523b1ee3849c98fb3625b9d1f9715, user_id=24304216b9ca483cbd21d493ae552929, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.613120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.638535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 796/1580] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:46 2022] PUT /identity/v3/projects/2dd523b1ee3849c98fb3625b9d1f9715/users/24304216b9ca483cbd21d493ae552929/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:46.642308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.681640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ff350-00d8-4cfe-8e53-c94ae3a7fe58 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.718908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:46.750807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7591-e989-44e5-a45a-b2ff8b290c01 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.768609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd33642-b4d9-433d-b06a-a7132f74ef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.769325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd33642-b4d9-433d-b06a-a7132f74ef44 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.769671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd33642-b4d9-433d-b06a-a7132f74ef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.770009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd33642-b4d9-433d-b06a-a7132f74ef44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.770721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd33642-b4d9-433d-b06a-a7132f74ef44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.772460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd33642-b4d9-433d-b06a-a7132f74ef44 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.787504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 797/1581] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:46.793393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.794096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.794443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.794797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.795770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ddc14-cfde-4ed5-a853-b00fe0bc23c5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.796729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ddc14-cfde-4ed5-a853-b00fe0bc23c5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.796729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ddc14-cfde-4ed5-a853-b00fe0bc23c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.818126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.822910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.845291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 785/1582] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:46.846418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e70ddc14-cfde-4ed5-a853-b00fe0bc23c5 None None] MFA Rules not processed for user `24304216b9ca483cbd21d493ae552929`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:46.847743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e70ddc14-cfde-4ed5-a853-b00fe0bc23c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:46.848717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.866049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c0627-36ae-45b7-8e89-9474f681eb27 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.884178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.884917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.885240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.885591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.887285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.890658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.912250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 786/1583] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:46.915038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:46.935804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:46.939952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e67a5-1ae6-4959-8365-b596ef4f0f58 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:46.950361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3078990-7171-44b5-98d5-902e46f74614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:46.951335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078990-7171-44b5-98d5-902e46f74614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:46.953005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078990-7171-44b5-98d5-902e46f74614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:46.953422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078990-7171-44b5-98d5-902e46f74614 admin admin] PATH_INFO: `/v3/projects/3aa7fb8ecfd8423480aa97000cad26ab/users/8a679ee275e5476b9816eb70362a20bc/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:46.988491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3078990-7171-44b5-98d5-902e46f74614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa7fb8ecfd8423480aa97000cad26ab, user_id=8a679ee275e5476b9816eb70362a20bc, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:46.993083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 798/1584] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:46.994207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3078990-7171-44b5-98d5-902e46f74614 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:46.997873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ddc14-cfde-4ed5-a853-b00fe0bc23c5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.018021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ddc14-cfde-4ed5-a853-b00fe0bc23c5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.021688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 787/1585] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:46 2022] PUT /identity/v3/projects/3aa7fb8ecfd8423480aa97000cad26ab/users/8a679ee275e5476b9816eb70362a20bc/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:47.033035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6082-7c8e-4697-8c3f-d5e7668a5bda None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.033293 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6082-7c8e-4697-8c3f-d5e7668a5bda None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.033570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6082-7c8e-4697-8c3f-d5e7668a5bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.091399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fc1e6082-7c8e-4697-8c3f-d5e7668a5bda None None] MFA Rules not processed for user `8a679ee275e5476b9816eb70362a20bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:47.092743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fc1e6082-7c8e-4697-8c3f-d5e7668a5bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:47.117823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.118319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.118828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.118932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.119549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.122150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.165967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 799/1586] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:46 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:47.167264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.216578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 788/1587] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:47 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:47.218091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6082-7c8e-4697-8c3f-d5e7668a5bda None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.246292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6082-7c8e-4697-8c3f-d5e7668a5bda None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.322368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89794496-194f-4e67-8045-3a8465059a94 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.335102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.335775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.336046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.336317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.336953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.339671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.354158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.354784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.355741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.356015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.356692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.358633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.378075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 800/1588] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:47 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:47.384375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.400708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c5983-305e-476a-9004-0e1867288dc4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.412777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 789/1589] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:47 2022] POST /identity/v3/projects => generated 409 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:47.414648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.415314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.416626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.417132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.417955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.418729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.420435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.423356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:47.438894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 801/1590] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:47.443836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.467736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff12a5-8389-4697-ae4e-7ac27ea2c3e7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.525582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9717-a72c-4391-a20c-1fb3d81843cd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.531587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.532357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.532684 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.533085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.534106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.536337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.580919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 790/1591] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:47 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:47.584757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.591306 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:47.610116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539312e-64ea-47b8-ace3-cd0df29a9a86 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.621760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.626333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.627313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.628296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.630193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.631940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.634456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.636700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.638863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.639252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] PATH_INFO: `/v3/projects/006bb8f1b0684f15b4e423940a83a597/users/21d81d34980b40a88ce81c6b93fbcc6c/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.659345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 791/1592] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:47.663541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.679200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52527806-a895-459c-9f58-b911dd2adcdb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.680313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006bb8f1b0684f15b4e423940a83a597, user_id=21d81d34980b40a88ce81c6b93fbcc6c, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.683200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.704311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.710028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.710333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.710615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] PATH_INFO: `/v3/projects/f4c8e8e7530e4a56a3ace2a214cae0c5/users/e444b3d8b37741448503650959b8c9db/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.716490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 802/1593] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:47 2022] PUT /identity/v3/projects/006bb8f1b0684f15b4e423940a83a597/users/21d81d34980b40a88ce81c6b93fbcc6c/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:47.717527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.770745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c8e8e7530e4a56a3ace2a214cae0c5, user_id=e444b3d8b37741448503650959b8c9db, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.773613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.801226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 792/1594] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:47 2022] PUT /identity/v3/projects/f4c8e8e7530e4a56a3ace2a214cae0c5/users/e444b3d8b37741448503650959b8c9db/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:47.801226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.803820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:47.840488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adac004-12c3-47fe-a834-1415edff7fb2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.899194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8bff0-d4f1-4a3d-8079-f282831f1a77 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.906984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.907888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.908300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.908696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] PATH_INFO: `/v3/projects/4293d6ff2cf548ae821148f9db4ba235` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.910016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4293d6ff2cf548ae821148f9db4ba235)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.912255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.924711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:47.925327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:47.925965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:47.926373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:47.927076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:47.929298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:47.946905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 803/1595] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:47.946905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:47.964135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747e795-ea62-4b6e-af72-fa6303c81666 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:47.970940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4293d6ff2cf548ae821148f9db4ba235', 'request_id': 'req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:47.979407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4293d6ff2cf548ae821148f9db4ba235', 'request_id': 'req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:47.981347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:47.996301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4293d6ff2cf548ae821148f9db4ba235', 'request_id': 'req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:48.010619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] The token cache is being invalidate because project 4293d6ff2cf548ae821148f9db4ba235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:48.010920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:48.011726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:48.013264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 793/1596] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:47 2022] DELETE /identity/v3/projects/4293d6ff2cf548ae821148f9db4ba235 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:48.017299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.018855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.056282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08b7a8-58b7-4310-ac71-32c244d0a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.056964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08b7a8-58b7-4310-ac71-32c244d0a656 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.057245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08b7a8-58b7-4310-ac71-32c244d0a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.057518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08b7a8-58b7-4310-ac71-32c244d0a656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.058764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08b7a8-58b7-4310-ac71-32c244d0a656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.060987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08b7a8-58b7-4310-ac71-32c244d0a656 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.073138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 804/1597] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:48.074796 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.075272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.075555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.086457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:48.099864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:48.101100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.111768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 805/1598] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:48.112718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.114000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.128087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580fa98-4f0f-43d4-a214-cd209dff462e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.134163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58cf3-acc4-4fa9-a000-8dfcf26c9d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.135100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58cf3-acc4-4fa9-a000-8dfcf26c9d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.135427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58cf3-acc4-4fa9-a000-8dfcf26c9d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.135702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58cf3-acc4-4fa9-a000-8dfcf26c9d16 admin admin] PATH_INFO: `/v3/projects/f4c8e8e7530e4a56a3ace2a214cae0c5/users/e444b3d8b37741448503650959b8c9db/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.139584 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd58cf3-acc4-4fa9-a000-8dfcf26c9d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c8e8e7530e4a56a3ace2a214cae0c5, user_id=e444b3d8b37741448503650959b8c9db, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.142998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd58cf3-acc4-4fa9-a000-8dfcf26c9d16 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.156594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 806/1599] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:48 2022] PUT /identity/v3/projects/f4c8e8e7530e4a56a3ace2a214cae0c5/users/e444b3d8b37741448503650959b8c9db/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:48.157884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f366e-a15c-4bd0-bcd3-acca11e33406 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.158168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f366e-a15c-4bd0-bcd3-acca11e33406 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.158442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f366e-a15c-4bd0-bcd3-acca11e33406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.162971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6f4c3-73d6-4b0b-8d8e-250f0f482bae admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.186608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-889f366e-a15c-4bd0-bcd3-acca11e33406 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:48.187988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-889f366e-a15c-4bd0-bcd3-acca11e33406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.275757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.277076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.277862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.278679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] PATH_INFO: `/v3/projects/006bb8f1b0684f15b4e423940a83a597/users/21d81d34980b40a88ce81c6b93fbcc6c/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.281327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=006bb8f1b0684f15b4e423940a83a597, user_id=21d81d34980b40a88ce81c6b93fbcc6c, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.281769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 807/1600] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:48.283064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0232bb-9330-4a86-be08-af1d248f0b1b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.283384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0232bb-9330-4a86-be08-af1d248f0b1b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.283658 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0232bb-9330-4a86-be08-af1d248f0b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.287830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.315903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 794/1601] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:48 2022] PUT /identity/v3/projects/006bb8f1b0684f15b4e423940a83a597/users/21d81d34980b40a88ce81c6b93fbcc6c/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:48.315903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.317806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5c0232bb-9330-4a86-be08-af1d248f0b1b None None] MFA Rules not processed for user `e444b3d8b37741448503650959b8c9db`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:48.318978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5c0232bb-9330-4a86-be08-af1d248f0b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.408490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f770-b46d-474f-83f4-468aaf89ed9e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.413784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.414383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.414656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.414924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.415594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:48.415908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.416057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 808/1602] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:48 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:48.418030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.418197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b24c93-bbef-4235-b18b-2d45c5955bc6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.418608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b24c93-bbef-4235-b18b-2d45c5955bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.419015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b24c93-bbef-4235-b18b-2d45c5955bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.436264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 795/1603] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:48.437208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.440376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.454583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f1b24c93-bbef-4235-b18b-2d45c5955bc6 None None] MFA Rules not processed for user `21d81d34980b40a88ce81c6b93fbcc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:48.456399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f1b24c93-bbef-4235-b18b-2d45c5955bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.471626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:48.518454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61602b-1bbf-4758-b89f-e31f4b0bb2df admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.531971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f2628-5bf0-4686-8a94-ee87fd09848d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.532654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f2628-5bf0-4686-8a94-ee87fd09848d service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.532929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f2628-5bf0-4686-8a94-ee87fd09848d service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.533522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f2628-5bf0-4686-8a94-ee87fd09848d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.537594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:48.546212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f2628-5bf0-4686-8a94-ee87fd09848d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.548974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f2628-5bf0-4686-8a94-ee87fd09848d service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.552833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 809/1604] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:48.553799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b24c93-bbef-4235-b18b-2d45c5955bc6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.562846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 796/1605] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:48 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:48.562846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523609cb-ec0c-455d-98b6-3a178232e9fe None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.563069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523609cb-ec0c-455d-98b6-3a178232e9fe None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.563235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523609cb-ec0c-455d-98b6-3a178232e9fe None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.564894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 797/1606] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:03:48.573440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523609cb-ec0c-455d-98b6-3a178232e9fe None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.579110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-523609cb-ec0c-455d-98b6-3a178232e9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.584126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b24c93-bbef-4235-b18b-2d45c5955bc6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.599028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da84327-420a-45e8-83a2-43c4cf7cd4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.602711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84327-420a-45e8-83a2-43c4cf7cd4ab admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.603855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84327-420a-45e8-83a2-43c4cf7cd4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.604202 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84327-420a-45e8-83a2-43c4cf7cd4ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.605141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da84327-420a-45e8-83a2-43c4cf7cd4ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.610463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523609cb-ec0c-455d-98b6-3a178232e9fe None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.611262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da84327-420a-45e8-83a2-43c4cf7cd4ab admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.622733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.623377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.623650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.623922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.626365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.678923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 810/1607] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:48 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:48.680415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4a504-1f74-49ec-bc6f-8b87f023ed9e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.680697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4a504-1f74-49ec-bc6f-8b87f023ed9e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.680992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4a504-1f74-49ec-bc6f-8b87f023ed9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.704573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d6a4a504-1f74-49ec-bc6f-8b87f023ed9e None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:48.707515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d6a4a504-1f74-49ec-bc6f-8b87f023ed9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:48.760619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.770049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.800306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 798/1608] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:48.802435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.807932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 811/1609] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:03:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:48.818138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83519909-fc20-46f7-ac59-edfce322aaad service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.862558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4a504-1f74-49ec-bc6f-8b87f023ed9e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:48.955862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.957040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.957511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.957964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.958069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4a504-1f74-49ec-bc6f-8b87f023ed9e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:48.959197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:48.963000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:48.963795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:48.963906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:48.964320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:48.964860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:48.966882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:49.001565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 799/1610] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:48 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:49.001565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:49.003912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:49.028111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0350c-0c90-4479-95b4-5946cfe26f3c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:49.034751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:49.035536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:49.035813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:49.036083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:49.046656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:49.048300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:49.059461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 800/1611] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:49.060306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:49.067483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:49.068985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:49.070706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae264b4-741b-4cb6-81cf-e29e3661dec1 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:49.076686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:49.077286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:49.077569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:49.077847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:49.078478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:49.080201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:49.093167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 801/1612] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:49.095969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 812/1613] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:48 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:03:49.102017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:49.114048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146bb0b-8608-481f-82da-06efec19883d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:49.118857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d789c-8b7f-422f-b641-f144cb54c6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:49.119875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d789c-8b7f-422f-b641-f144cb54c6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:49.119875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d789c-8b7f-422f-b641-f144cb54c6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:49.120063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d789c-8b7f-422f-b641-f144cb54c6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:49.120694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d789c-8b7f-422f-b641-f144cb54c6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:49.122307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d789c-8b7f-422f-b641-f144cb54c6f8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:49.138691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 802/1614] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:49.151663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:49.172382 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b0cbb-9063-4a18-bff7-38d53989dd93 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:49.181479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:49.182354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:49.182934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:49.183475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] PATH_INFO: `/v3/projects/2f61d3c56dbe430885109d5b42b0c272/users/a3a10f7f14e24e49adb84359a5f309b1/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:49.207863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f61d3c56dbe430885109d5b42b0c272, user_id=a3a10f7f14e24e49adb84359a5f309b1, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:49.210633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:49.214086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:49.243776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 813/1615] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:49 2022] PUT /identity/v3/projects/2f61d3c56dbe430885109d5b42b0c272/users/a3a10f7f14e24e49adb84359a5f309b1/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:49.246955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e2c72-5159-4433-af35-eda94a6449e0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:49.247347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e2c72-5159-4433-af35-eda94a6449e0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:49.247621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e2c72-5159-4433-af35-eda94a6449e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:49.298924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f98e2c72-5159-4433-af35-eda94a6449e0 None None] MFA Rules not processed for user `a3a10f7f14e24e49adb84359a5f309b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:49.301288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f98e2c72-5159-4433-af35-eda94a6449e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:49.310109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:49.469531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 803/1616] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:49.480210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:49.481385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:49.589596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8d83-067c-4dc0-8163-510220ac7bbc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:49.595860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caeda8-8923-4728-8c71-7ab9af1482a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:49.596481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caeda8-8923-4728-8c71-7ab9af1482a1 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:49.596754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caeda8-8923-4728-8c71-7ab9af1482a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:49.597322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caeda8-8923-4728-8c71-7ab9af1482a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:49.612681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07caeda8-8923-4728-8c71-7ab9af1482a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:49.614503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07caeda8-8923-4728-8c71-7ab9af1482a1 service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:49.631563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 814/1617] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:51.496476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e2c72-5159-4433-af35-eda94a6449e0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:51.558910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e2c72-5159-4433-af35-eda94a6449e0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:51.563470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:51.564120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:51.564398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:51.564670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] PATH_INFO: `/v3/users/24304216b9ca483cbd21d493ae552929` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:51.565852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24304216b9ca483cbd21d493ae552929)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:51.568056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:51.617046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24304216b9ca483cbd21d493ae552929'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:51.617756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24304216b9ca483cbd21d493ae552929'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:51.625433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24304216b9ca483cbd21d493ae552929'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:51.632557 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24304216b9ca483cbd21d493ae552929'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:51.633297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24304216b9ca483cbd21d493ae552929'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:51.651714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 804/1618] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:51 2022] DELETE /identity/v3/users/24304216b9ca483cbd21d493ae552929 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:51.656396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:51.656883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:51.656969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:51.675776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:51.676891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:51.737300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 815/1619] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:51.748430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:51.749858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:51.780792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:51.809170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14673599-ea87-47a2-8ba2-af25d7f594f8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:51.812769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:51.813374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:51.813652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:51.813926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:51.822776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:51.824561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:51.833880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 805/1620] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:03:52.166900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:52.171318 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:52.191172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77051b-ab64-4d0a-b29f-5255d5afd105 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:52.196532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:52.213304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:52.213710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:52.215128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] PATH_INFO: `/v3/projects/2dd523b1ee3849c98fb3625b9d1f9715` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:52.217289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd523b1ee3849c98fb3625b9d1f9715)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:52.220449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:52.288800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd523b1ee3849c98fb3625b9d1f9715', 'request_id': 'req-d347f51b-1895-4a1c-9f6e-ae520643d9f2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.294830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd523b1ee3849c98fb3625b9d1f9715', 'request_id': 'req-d347f51b-1895-4a1c-9f6e-ae520643d9f2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.301097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd523b1ee3849c98fb3625b9d1f9715', 'request_id': 'req-d347f51b-1895-4a1c-9f6e-ae520643d9f2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.313088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] The token cache is being invalidate because project 2dd523b1ee3849c98fb3625b9d1f9715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:52.313369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.313979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.315500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 816/1621] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:52 2022] DELETE /identity/v3/projects/2dd523b1ee3849c98fb3625b9d1f9715 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:52.319473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:52.329225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463613b-123c-4f34-8ad9-11173def00e5 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:52.384056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:52.384677 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:52.384952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:52.385221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] PATH_INFO: `/v3/users/21d81d34980b40a88ce81c6b93fbcc6c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:52.386350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d81d34980b40a88ce81c6b93fbcc6c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:52.388318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:52.433921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d81d34980b40a88ce81c6b93fbcc6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.434497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d81d34980b40a88ce81c6b93fbcc6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.441380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d81d34980b40a88ce81c6b93fbcc6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.449236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d81d34980b40a88ce81c6b93fbcc6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.449885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-660dc38f-553a-43b1-960a-5e71fd3dacde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d81d34980b40a88ce81c6b93fbcc6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.467390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 806/1622] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:03:52 2022] DELETE /identity/v3/users/21d81d34980b40a88ce81c6b93fbcc6c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:52.760708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:52.762232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:52.770562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:52.787682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f51b-1895-4a1c-9f6e-ae520643d9f2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:52.842980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:52.843706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:52.843989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:52.844262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] PATH_INFO: `/v3/projects/006bb8f1b0684f15b4e423940a83a597` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:52.845360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=006bb8f1b0684f15b4e423940a83a597)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:52.847311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:52.895054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006bb8f1b0684f15b4e423940a83a597', 'request_id': 'req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.900520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006bb8f1b0684f15b4e423940a83a597', 'request_id': 'req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.906685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006bb8f1b0684f15b4e423940a83a597', 'request_id': 'req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.919094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] The token cache is being invalidate because project 006bb8f1b0684f15b4e423940a83a597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:03:52.919403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.920025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-55b6cc0b-4e80-4f86-8f92-513ca1b83c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:03:52.921497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 817/1623] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:03:52 2022] DELETE /identity/v3/projects/006bb8f1b0684f15b4e423940a83a597 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:03:52.926439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d4901-4516-4c31-9f10-f1630cf8c921 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:52.926884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d4901-4516-4c31-9f10-f1630cf8c921 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:52.926984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d4901-4516-4c31-9f10-f1630cf8c921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:52.947007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-da4d4901-4516-4c31-9f10-f1630cf8c921 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:52.948174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-da4d4901-4516-4c31-9f10-f1630cf8c921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:52.998235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:53.019298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 807/1624] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:53.027540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a5447-1bff-40a5-bce9-8f4bd2c49728 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.027825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a5447-1bff-40a5-bce9-8f4bd2c49728 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.028100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a5447-1bff-40a5-bce9-8f4bd2c49728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.047924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1e4a5447-1bff-40a5-bce9-8f4bd2c49728 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:53.049131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1e4a5447-1bff-40a5-bce9-8f4bd2c49728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:53.056956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 818/1625] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:53.061207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d4901-4516-4c31-9f10-f1630cf8c921 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:53.070645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d4901-4516-4c31-9f10-f1630cf8c921 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:53.075434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:53.076022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.076313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.076569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.077198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:03:53.077496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:53.079079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:53.090917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 808/1626] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:03:53.095052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a5447-1bff-40a5-bce9-8f4bd2c49728 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:53.104044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a5447-1bff-40a5-bce9-8f4bd2c49728 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:53.109160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:53.109746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.110029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.110301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.110927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:53.112871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:53.140130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 819/1627] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:03:53 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:03:53.144316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:53.155105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e981a7-dc7d-4768-b59b-70590a700c78 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:53.211085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:53.211633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.211910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.212176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.212787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:53.214718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:53.237157 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 809/1628] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:03:53 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:03:53.238315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:53.249123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678730-c46e-4896-8a0d-10785f8efcc9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:53.253662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:53.254254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.254558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.254831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.255524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:53.257102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:53.270516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 820/1629] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:53.271913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:53.282331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bfa9-616d-430a-9ee4-4e80da151b95 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:53.289815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f81ac5-930b-4788-8475-3611d304dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:53.289815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81ac5-930b-4788-8475-3611d304dc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.289815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81ac5-930b-4788-8475-3611d304dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.289815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81ac5-930b-4788-8475-3611d304dc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.290212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f81ac5-930b-4788-8475-3611d304dc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:53.291961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f81ac5-930b-4788-8475-3611d304dc04 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:53.303254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 810/1630] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:03:53.307491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:53.319893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669d4a6-90da-4e00-80b8-0bd519b621a8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:53.331187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:53.334028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.334468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.335415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] PATH_INFO: `/v3/projects/1fd3f16846a2498ba61ac29a2df44957/users/50d81a392850498f9cb6bf2bdd6e97bd/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.359296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd3f16846a2498ba61ac29a2df44957, user_id=50d81a392850498f9cb6bf2bdd6e97bd, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:53.361993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:53.375445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 821/1631] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:03:53 2022] PUT /identity/v3/projects/1fd3f16846a2498ba61ac29a2df44957/users/50d81a392850498f9cb6bf2bdd6e97bd/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:03:53.387371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.387654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.387923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.416855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] MFA Rules not processed for user `50d81a392850498f9cb6bf2bdd6e97bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:03:53.418698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:53.487380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 811/1632] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:03:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:03:53.498601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:03:53.500460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:03:53.534461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:03:53.562216 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dd364-418b-42c9-b8b9-c57e38320137 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:03:53.565878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c175a-9460-4344-bf2f-163ab2f2e43a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:03:53.566479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c175a-9460-4344-bf2f-163ab2f2e43a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:03:53.566758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c175a-9460-4344-bf2f-163ab2f2e43a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:03:53.567030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c175a-9460-4344-bf2f-163ab2f2e43a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:03:53.575454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c175a-9460-4344-bf2f-163ab2f2e43a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:03:53.576880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c175a-9460-4344-bf2f-163ab2f2e43a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:03:53.585679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 822/1633] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:03:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:00.308662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:00.310862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:00.381691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:00.407815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40de32-85cb-47fe-866e-290148aee2a1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:00.413231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:00.413900 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:00.414237 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:00.414434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] PATH_INFO: `/v3/users/8a679ee275e5476b9816eb70362a20bc` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:00.415614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a679ee275e5476b9816eb70362a20bc)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:00.418295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:00.504306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a679ee275e5476b9816eb70362a20bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:00.504973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a679ee275e5476b9816eb70362a20bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:00.513727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a679ee275e5476b9816eb70362a20bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:00.524523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a679ee275e5476b9816eb70362a20bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:00.525159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a679ee275e5476b9816eb70362a20bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:00.561601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 812/1634] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:00 2022] DELETE /identity/v3/users/8a679ee275e5476b9816eb70362a20bc => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:00.565914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:00.566412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:00.566633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:00.586676 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:00.588044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:00.693806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 823/1635] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:00.704740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:00.706361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:00.778872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97840a9c-e5a2-4c6c-9356-bf8639da42f7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:00.782958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:00.783679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:00.784027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:00.784360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:00.798515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:00.798515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:00.815657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 813/1636] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:01.123791 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:01.127357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:01.164373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5873b-9bf7-4e66-a8f5-1f38d96e8f59 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:01.170084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:01.170727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:01.171008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:01.171343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] PATH_INFO: `/v3/projects/3aa7fb8ecfd8423480aa97000cad26ab` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:01.172428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa7fb8ecfd8423480aa97000cad26ab)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:01.174432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:01.233817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa7fb8ecfd8423480aa97000cad26ab', 'request_id': 'req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.239461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa7fb8ecfd8423480aa97000cad26ab', 'request_id': 'req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.246441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa7fb8ecfd8423480aa97000cad26ab', 'request_id': 'req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.259774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] The token cache is being invalidate because project 3aa7fb8ecfd8423480aa97000cad26ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:01.260059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.260695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.262933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 824/1637] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:01 2022] DELETE /identity/v3/projects/3aa7fb8ecfd8423480aa97000cad26ab => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:01.266771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:01.276848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ef6e-7c25-42af-a2b2-292199c4af6b service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:01.365611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:01.366707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:01.367085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:01.367596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] PATH_INFO: `/v3/users/e444b3d8b37741448503650959b8c9db` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:01.373342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e444b3d8b37741448503650959b8c9db)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:01.373342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:01.378586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:01.424115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e444b3d8b37741448503650959b8c9db'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.424938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e444b3d8b37741448503650959b8c9db'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.433228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e444b3d8b37741448503650959b8c9db'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.441731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e444b3d8b37741448503650959b8c9db'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.442564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47d3a316-504f-4b7a-88f9-48be932168f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e444b3d8b37741448503650959b8c9db'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.469405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 814/1638] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:01 2022] DELETE /identity/v3/users/e444b3d8b37741448503650959b8c9db => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:01.772924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:01.774616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:01.789944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2977073-c1ed-4b2b-94cb-fbe1bfde994d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:01.856076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:01.870136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:01.870805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:01.871083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:01.871383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] PATH_INFO: `/v3/projects/f4c8e8e7530e4a56a3ace2a214cae0c5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:01.872497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c8e8e7530e4a56a3ace2a214cae0c5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:01.874387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:01.934117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c8e8e7530e4a56a3ace2a214cae0c5', 'request_id': 'req-37f49b29-e454-4848-aee0-ab65a8da92ee'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.938073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c8e8e7530e4a56a3ace2a214cae0c5', 'request_id': 'req-37f49b29-e454-4848-aee0-ab65a8da92ee'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.944599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c8e8e7530e4a56a3ace2a214cae0c5', 'request_id': 'req-37f49b29-e454-4848-aee0-ab65a8da92ee'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.960925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] The token cache is being invalidate because project f4c8e8e7530e4a56a3ace2a214cae0c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:01.961283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.962025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37f49b29-e454-4848-aee0-ab65a8da92ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:01.963617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 825/1639] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:01 2022] DELETE /identity/v3/projects/f4c8e8e7530e4a56a3ace2a214cae0c5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:01.968943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724cb4-ff22-4267-852f-bb06a5ba7386 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:01.969293 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724cb4-ff22-4267-852f-bb06a5ba7386 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:01.969642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724cb4-ff22-4267-852f-bb06a5ba7386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:01.992053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e2724cb4-ff22-4267-852f-bb06a5ba7386 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:01.993299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e2724cb4-ff22-4267-852f-bb06a5ba7386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:02.067905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 815/1640] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:02.076620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657f11-c07d-4fc7-9042-3b4882d7304e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.076902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657f11-c07d-4fc7-9042-3b4882d7304e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.077175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657f11-c07d-4fc7-9042-3b4882d7304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.096659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a2657f11-c07d-4fc7-9042-3b4882d7304e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:02.097908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a2657f11-c07d-4fc7-9042-3b4882d7304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:02.105903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 826/1641] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:02.110056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2724cb4-ff22-4267-852f-bb06a5ba7386 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.120816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2724cb4-ff22-4267-852f-bb06a5ba7386 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.125488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.126111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.126385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.126657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.127319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:02.127621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.129248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.140861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 816/1642] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:02.145137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2657f11-c07d-4fc7-9042-3b4882d7304e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.156240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2657f11-c07d-4fc7-9042-3b4882d7304e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.161437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.162046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.162328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.162594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.163235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.165144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.198668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 827/1643] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:02 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:02.200012 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.210080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fd053-6efc-4725-ada2-4ea07853f5b8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.272929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.273547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.273846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.274118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.274804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.277396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.301648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 817/1644] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:02 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:02.302877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.314342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2916a86-5b85-4f67-ab42-72d6881b989d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.319509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.320118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.320392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.321022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.321682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.324663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.340513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 828/1645] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:02.341220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.347109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:02.354337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53963345-7855-478d-b8bb-e5f3c2a17399 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.361518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b1c0a-8661-49df-987f-76f74024507e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.362567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b1c0a-8661-49df-987f-76f74024507e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.363870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b1c0a-8661-49df-987f-76f74024507e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.364422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b1c0a-8661-49df-987f-76f74024507e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.365192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b1c0a-8661-49df-987f-76f74024507e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.367804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b1c0a-8661-49df-987f-76f74024507e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.392391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 818/1646] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:02.393632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.405571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7e001-90c5-4cbf-b93c-69f733f13589 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.411908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.412965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.412965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.413227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] PATH_INFO: `/v3/projects/333c6310a19742f2894639951d979a74/users/4c4320edd17e4ae1b2e983bc311ae889/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.444752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333c6310a19742f2894639951d979a74, user_id=4c4320edd17e4ae1b2e983bc311ae889, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.448020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.470993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 829/1647] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:02 2022] PUT /identity/v3/projects/333c6310a19742f2894639951d979a74/users/4c4320edd17e4ae1b2e983bc311ae889/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:02.483321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a65c7-35d9-40b0-b57c-6452571d2b47 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.483321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a65c7-35d9-40b0-b57c-6452571d2b47 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.483321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a65c7-35d9-40b0-b57c-6452571d2b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.528935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cb7a65c7-35d9-40b0-b57c-6452571d2b47 None None] MFA Rules not processed for user `4c4320edd17e4ae1b2e983bc311ae889`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:02.530969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cb7a65c7-35d9-40b0-b57c-6452571d2b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:02.681499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 819/1648] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:02.686241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.758753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeb21b-7c30-414e-a6d6-9767c2b74412 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.766365 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.767140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.767505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.767839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.768542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.770523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.798806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 830/1649] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:02 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:02.802750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a65c7-35d9-40b0-b57c-6452571d2b47 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.814231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a65c7-35d9-40b0-b57c-6452571d2b47 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.878164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.878786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.879060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.879377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.880007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.881935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.911047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 820/1650] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:02 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:02.914923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.929274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85289-eb63-4c96-90a5-52d096badbf3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.936402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.937012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.937292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.937568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.938201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.939846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:02.941324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:02.952913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 831/1651] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:02.958285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:02.967604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51a53-4da5-48a6-a378-770bdbecdbb0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:02.972143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:02.972795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:02.973071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:02.973340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] PATH_INFO: `/v3/projects/116248ba54344668b8e5fc328f0c771d/users/b1c37c434cb24a4283078847713e1056/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:02.988424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116248ba54344668b8e5fc328f0c771d, user_id=b1c37c434cb24a4283078847713e1056, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:02.991074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:03.004428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 821/1652] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:02 2022] PUT /identity/v3/projects/116248ba54344668b8e5fc328f0c771d/users/b1c37c434cb24a4283078847713e1056/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:03.008663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:03.019355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1154a-83c4-410a-aaec-497c27104e65 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:03.089446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a47e7-ec6d-439d-b47f-67fe06eecaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:03.090225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a47e7-ec6d-439d-b47f-67fe06eecaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:03.090484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a47e7-ec6d-439d-b47f-67fe06eecaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:03.090838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a47e7-ec6d-439d-b47f-67fe06eecaa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:03.091553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a47e7-ec6d-439d-b47f-67fe06eecaa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:03.093703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a47e7-ec6d-439d-b47f-67fe06eecaa8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:03.105271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 832/1653] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:03.109291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:03.122156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffdad4-fafd-4aa7-8f72-c50f06a16e62 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:03.127155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:03.128016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:03.128369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:03.128758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] PATH_INFO: `/v3/projects/116248ba54344668b8e5fc328f0c771d/users/b1c37c434cb24a4283078847713e1056/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:03.132043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116248ba54344668b8e5fc328f0c771d, user_id=b1c37c434cb24a4283078847713e1056, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:03.135025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:03.152522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 822/1654] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:03 2022] PUT /identity/v3/projects/116248ba54344668b8e5fc328f0c771d/users/b1c37c434cb24a4283078847713e1056/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:03.163672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:03.163963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:03.164239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:03.199377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] MFA Rules not processed for user `b1c37c434cb24a4283078847713e1056`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:03.200737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:03.305558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 833/1655] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:03.318513 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:03.327331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:03.369476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:03.399621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee201491-3368-4fd9-9a44-01ca44360f70 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:03.403613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:03.404345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:03.404644 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:03.404920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:03.416391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:03.418453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:03.431369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 823/1656] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:03.556309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:03.558032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:03.571561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145956-9232-47ac-9cfb-c81cc0dfff0a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:03.575913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272750e-9b6f-4e89-82ed-afb1c6eb4770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:03.576539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272750e-9b6f-4e89-82ed-afb1c6eb4770 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:03.576824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272750e-9b6f-4e89-82ed-afb1c6eb4770 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:03.577096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272750e-9b6f-4e89-82ed-afb1c6eb4770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:03.667707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272750e-9b6f-4e89-82ed-afb1c6eb4770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:03.669417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272750e-9b6f-4e89-82ed-afb1c6eb4770 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:03.681644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 834/1657] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:04.039499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:04.041200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:04.168737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a42441-6896-49b8-a3c0-1680ec4fdfc8 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:04.180964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:04.181724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:04.182310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:04.182760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] PATH_INFO: `/v3/users/dee6061675c547ccb231083fc7228bc1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:04.190334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=dee6061675c547ccb231083fc7228bc1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:04.192550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:04.286491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dee6061675c547ccb231083fc7228bc1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:04.288557 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dee6061675c547ccb231083fc7228bc1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:04.297085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dee6061675c547ccb231083fc7228bc1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:04.308251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dee6061675c547ccb231083fc7228bc1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:04.309068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dee6061675c547ccb231083fc7228bc1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:04.335292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 824/1658] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:04 2022] DELETE /identity/v3/users/dee6061675c547ccb231083fc7228bc1 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:04.337687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:04.338153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:04.338244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:04.366741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:04.367779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:04.379665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:04.440167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 835/1659] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:04.454392 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:04.458077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:04.527931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc705322-b3ca-4a94-89e0-71abcac608ef admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:04.531567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:04.532267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:04.532610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:04.532953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:04.542399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:04.544025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:04.550486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:04.554981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 825/1660] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:05.443015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:05.445162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:05.461372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ea1d5-626d-4e23-9e7d-41639e66d742 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:05.468152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:05.469444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:05.469977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:05.470504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] PATH_INFO: `/v3/projects/e6317c258bda49789a5ab783dfd0dedc` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:05.472005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6317c258bda49789a5ab783dfd0dedc)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:05.474220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:05.546902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6317c258bda49789a5ab783dfd0dedc', 'request_id': 'req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.553811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6317c258bda49789a5ab783dfd0dedc', 'request_id': 'req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.565644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6317c258bda49789a5ab783dfd0dedc', 'request_id': 'req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.580187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] The token cache is being invalidate because project e6317c258bda49789a5ab783dfd0dedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:05.580559 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.581197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.582805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 836/1661] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:05 2022] DELETE /identity/v3/projects/e6317c258bda49789a5ab783dfd0dedc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:05.586886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:05.596981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639c69-811b-4684-9191-9c89dbe8015e service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:05.714738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:05.716163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:05.716632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:05.716762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] PATH_INFO: `/v3/users/d44529781ca847bc8d57f99d38a3bee2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:05.718320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44529781ca847bc8d57f99d38a3bee2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:05.720647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:05.775164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44529781ca847bc8d57f99d38a3bee2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.776068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44529781ca847bc8d57f99d38a3bee2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.783282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44529781ca847bc8d57f99d38a3bee2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.790539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44529781ca847bc8d57f99d38a3bee2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.791292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-824e0307-f74b-4ae2-9d2b-1d62be0214aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44529781ca847bc8d57f99d38a3bee2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:05.810466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 826/1662] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:05 2022] DELETE /identity/v3/users/d44529781ca847bc8d57f99d38a3bee2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:06.111923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:06.113379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:06.127178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b15e4-45d4-4e36-b683-1481ce5ec3ed admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:06.194627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:06.195303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.195586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.195861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] PATH_INFO: `/v3/projects/a29e4301528e443ca019de9de38d52da` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.197030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29e4301528e443ca019de9de38d52da)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:06.199249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:06.254348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:06.266851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29e4301528e443ca019de9de38d52da', 'request_id': 'req-d2582698-176e-4d7a-81cd-d076e2f046f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:06.274051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29e4301528e443ca019de9de38d52da', 'request_id': 'req-d2582698-176e-4d7a-81cd-d076e2f046f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:06.281246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29e4301528e443ca019de9de38d52da', 'request_id': 'req-d2582698-176e-4d7a-81cd-d076e2f046f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:06.299497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] The token cache is being invalidate because project a29e4301528e443ca019de9de38d52da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:06.299497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:06.299497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d2582698-176e-4d7a-81cd-d076e2f046f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:06.300052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 837/1663] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:06 2022] DELETE /identity/v3/projects/a29e4301528e443ca019de9de38d52da => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:06.304877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ee022-1390-4844-a2c5-35c7ea7eb5b2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.305239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ee022-1390-4844-a2c5-35c7ea7eb5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.305851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ee022-1390-4844-a2c5-35c7ea7eb5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.330058 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e37ee022-1390-4844-a2c5-35c7ea7eb5b2 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:06.331306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e37ee022-1390-4844-a2c5-35c7ea7eb5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:06.448524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 827/1664] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:06.456809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2dca0-e187-46ff-9b50-3da00f26097e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.457310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2dca0-e187-46ff-9b50-3da00f26097e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.457413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2dca0-e187-46ff-9b50-3da00f26097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.479075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dff2dca0-e187-46ff-9b50-3da00f26097e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:06.480516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dff2dca0-e187-46ff-9b50-3da00f26097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:06.489614 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 838/1665] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:06.493988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ee022-1390-4844-a2c5-35c7ea7eb5b2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:06.510051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ee022-1390-4844-a2c5-35c7ea7eb5b2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:06.516208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:06.517121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.517121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.517380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.519051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:06.519051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:06.520114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:06.532667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 828/1666] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:06.536983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2dca0-e187-46ff-9b50-3da00f26097e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:06.546340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2dca0-e187-46ff-9b50-3da00f26097e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:06.551542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:06.552204 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.552518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.552893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.553853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:06.555689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:06.588340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 839/1667] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:06 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:06.592657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:06.600658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:06.604834 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a948a8-8029-44bc-87aa-1d06babf54ce admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:06.670069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:06.670683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.671039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.671233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.671882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:06.673931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:06.699464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 829/1668] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:06 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:06.703353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:06.721443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff01d0-43bc-4abd-bdb4-2efbfab33b46 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:06.738730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:06.739777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.740036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.740701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.742955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:06.744753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:06.762662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 840/1669] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:06.763862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:06.779826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b84ff-7ba6-4ba7-bf4c-9fa69c91a307 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:06.785689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13693c-2f43-4ddd-b49d-da8228d040a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:06.786359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13693c-2f43-4ddd-b49d-da8228d040a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.786704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13693c-2f43-4ddd-b49d-da8228d040a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.787044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13693c-2f43-4ddd-b49d-da8228d040a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.788201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13693c-2f43-4ddd-b49d-da8228d040a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:06.791747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13693c-2f43-4ddd-b49d-da8228d040a8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:06.816145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 830/1670] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:06.817928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:06.833177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e970-f6f9-4db2-b223-e8ca1db1b70c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:06.841741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:06.842542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.842844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.843118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] PATH_INFO: `/v3/projects/4e0680c2e78b46a8b946d68f0a26f5d4/users/af0d79ae883d43a59af2508ea62000e9/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.861243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0680c2e78b46a8b946d68f0a26f5d4, user_id=af0d79ae883d43a59af2508ea62000e9, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:06.864132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:06.889331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 841/1671] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:06 2022] PUT /identity/v3/projects/4e0680c2e78b46a8b946d68f0a26f5d4/users/af0d79ae883d43a59af2508ea62000e9/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:06.901734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2b54-6913-48b0-8ea3-7a4a305375a6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:06.902466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2b54-6913-48b0-8ea3-7a4a305375a6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:06.902885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2b54-6913-48b0-8ea3-7a4a305375a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:06.955136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a97a2b54-6913-48b0-8ea3-7a4a305375a6 None None] MFA Rules not processed for user `af0d79ae883d43a59af2508ea62000e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:06.955136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a97a2b54-6913-48b0-8ea3-7a4a305375a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:07.059199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 831/1672] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:07.059435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:07.186644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4909da-d5ec-480e-87dc-adbb51f06af2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:07.193397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:07.194186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.194525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.200820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.202164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:07.204694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:07.251141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 842/1673] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:07 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:07.255247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a2b54-6913-48b0-8ea3-7a4a305375a6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:07.266975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a2b54-6913-48b0-8ea3-7a4a305375a6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:07.408683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:07.409576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.410117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.410737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.411637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:07.413788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:07.451498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 832/1674] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:07 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:07.455274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:07.467606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b64f1-40a4-48d2-ba1f-e97ff99c5403 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:07.473661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:07.474344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.474808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.475170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.476001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:07.483922 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:07.496853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 843/1675] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:07.501356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:07.506936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:07.525759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922469f1-a622-4dd5-b66c-c0d329ba0b4d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:07.525759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:07.525759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.525759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.529177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] PATH_INFO: `/v3/projects/fb3f8d8ac03e45d0976048d68539bd82/users/f82e6f3709c0452d8e32a897a35092ad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.543111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3f8d8ac03e45d0976048d68539bd82, user_id=f82e6f3709c0452d8e32a897a35092ad, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:07.547161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:07.566036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 833/1676] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:07 2022] PUT /identity/v3/projects/fb3f8d8ac03e45d0976048d68539bd82/users/f82e6f3709c0452d8e32a897a35092ad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:07.570070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:07.587948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d13b5-9a99-4bd8-b46b-588c0e715cc7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:07.634611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:07.660057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424dec3-e50f-4075-9b0c-f3a06bcd82f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:07.660834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424dec3-e50f-4075-9b0c-f3a06bcd82f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.661752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424dec3-e50f-4075-9b0c-f3a06bcd82f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.662044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424dec3-e50f-4075-9b0c-f3a06bcd82f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.662693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424dec3-e50f-4075-9b0c-f3a06bcd82f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:07.664490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424dec3-e50f-4075-9b0c-f3a06bcd82f7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:07.676170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 844/1677] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:07.679994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:07.691004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa3d15-86bb-4b54-a12e-c7eba5e0b598 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:07.696888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:07.697656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.697899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.698231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] PATH_INFO: `/v3/projects/fb3f8d8ac03e45d0976048d68539bd82/users/f82e6f3709c0452d8e32a897a35092ad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.700807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3f8d8ac03e45d0976048d68539bd82, user_id=f82e6f3709c0452d8e32a897a35092ad, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:07.704263 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:07.717500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 834/1678] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:07 2022] PUT /identity/v3/projects/fb3f8d8ac03e45d0976048d68539bd82/users/f82e6f3709c0452d8e32a897a35092ad/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:07.730774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.731520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.731520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.762635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] MFA Rules not processed for user `f82e6f3709c0452d8e32a897a35092ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:07.763889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:07.854286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 845/1679] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:07.866316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:07.868108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:07.946451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cdd55-da00-4fc7-877d-a28c78c5a5a9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:07.951635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:07.952227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:07.952503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:07.952773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:07.971306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:07.973005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:07.988087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 835/1680] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:08.075810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:08.077234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:08.091878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fc5d9-937a-4dc7-88ad-d629f736e7ca None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:08.095558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca87e1-c3f8-47a6-9d0e-d8824e231dde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:08.096173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca87e1-c3f8-47a6-9d0e-d8824e231dde service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:08.096450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca87e1-c3f8-47a6-9d0e-d8824e231dde service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:08.097092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca87e1-c3f8-47a6-9d0e-d8824e231dde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:08.161137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca87e1-c3f8-47a6-9d0e-d8824e231dde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:08.162590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca87e1-c3f8-47a6-9d0e-d8824e231dde service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:08.173347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 846/1681] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:09.218388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:09.219974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:09.290017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f6130-f781-4dc3-8cca-bfc6eae35fb7 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:09.295545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:09.297174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:09.297174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:09.297174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] PATH_INFO: `/v3/users/4c4320edd17e4ae1b2e983bc311ae889` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:09.298040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c4320edd17e4ae1b2e983bc311ae889)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:09.299984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:09.379321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c4320edd17e4ae1b2e983bc311ae889'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:09.380352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c4320edd17e4ae1b2e983bc311ae889'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:09.390446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c4320edd17e4ae1b2e983bc311ae889'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:09.409567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c4320edd17e4ae1b2e983bc311ae889'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:09.411314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c4320edd17e4ae1b2e983bc311ae889'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:09.428497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:09.445133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 836/1682] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:09 2022] DELETE /identity/v3/users/4c4320edd17e4ae1b2e983bc311ae889 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:09.449369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571af304-899b-418f-9ebd-83184d969fab None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:09.450663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571af304-899b-418f-9ebd-83184d969fab None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:09.451154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571af304-899b-418f-9ebd-83184d969fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:09.485781 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-571af304-899b-418f-9ebd-83184d969fab None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:09.488047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-571af304-899b-418f-9ebd-83184d969fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:09.566866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:09.580735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 847/1683] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:09.591253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:09.592901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:09.706755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8a959-4a7f-446a-9a0c-f128870fa1b7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:09.711000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:09.711904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:09.712179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:09.712450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:09.727801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:09.732667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:09.748434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 837/1684] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:10.363722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571af304-899b-418f-9ebd-83184d969fab None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:10.365433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-571af304-899b-418f-9ebd-83184d969fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:10.383239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571af304-899b-418f-9ebd-83184d969fab None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:10.388943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:10.389693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:10.389973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:10.390279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] PATH_INFO: `/v3/projects/333c6310a19742f2894639951d979a74` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:10.391369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=333c6310a19742f2894639951d979a74)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:10.393325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:10.441244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333c6310a19742f2894639951d979a74', 'request_id': 'req-3a48ec67-d69b-4c7a-8570-32c511740322'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.446519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333c6310a19742f2894639951d979a74', 'request_id': 'req-3a48ec67-d69b-4c7a-8570-32c511740322'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.452848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333c6310a19742f2894639951d979a74', 'request_id': 'req-3a48ec67-d69b-4c7a-8570-32c511740322'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.465681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] The token cache is being invalidate because project 333c6310a19742f2894639951d979a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:10.465968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.466967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.468494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 848/1685] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:10 2022] DELETE /identity/v3/projects/333c6310a19742f2894639951d979a74 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:10.472442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:10.483422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482a7be-b7de-40fb-9ed3-b054016b8380 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:10.538800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:10.539447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:10.539721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:10.539988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] PATH_INFO: `/v3/users/b1c37c434cb24a4283078847713e1056` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:10.541090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c37c434cb24a4283078847713e1056)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:10.543574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:10.608536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c37c434cb24a4283078847713e1056'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.609192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c37c434cb24a4283078847713e1056'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.616106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c37c434cb24a4283078847713e1056'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.624235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c37c434cb24a4283078847713e1056'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.624829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82f0017e-5798-48db-ba29-54f2761aec6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c37c434cb24a4283078847713e1056'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:10.652670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 838/1686] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:10 2022] DELETE /identity/v3/users/b1c37c434cb24a4283078847713e1056 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:10.939141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:10.940558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:10.955991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48ec67-d69b-4c7a-8570-32c511740322 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.014211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.014873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.015160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.015466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] PATH_INFO: `/v3/projects/116248ba54344668b8e5fc328f0c771d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.016579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116248ba54344668b8e5fc328f0c771d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.018487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.069123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116248ba54344668b8e5fc328f0c771d', 'request_id': 'req-544027bb-53b4-45be-bef7-29fa15cf9946'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:11.076103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116248ba54344668b8e5fc328f0c771d', 'request_id': 'req-544027bb-53b4-45be-bef7-29fa15cf9946'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:11.083180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116248ba54344668b8e5fc328f0c771d', 'request_id': 'req-544027bb-53b4-45be-bef7-29fa15cf9946'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:11.098074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] The token cache is being invalidate because project 116248ba54344668b8e5fc328f0c771d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:11.098377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:11.099026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-544027bb-53b4-45be-bef7-29fa15cf9946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:11.100620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 849/1687] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:10 2022] DELETE /identity/v3/projects/116248ba54344668b8e5fc328f0c771d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:11.108408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2f3d-b74c-40ab-9e97-2302e557281b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.108687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2f3d-b74c-40ab-9e97-2302e557281b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.108956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2f3d-b74c-40ab-9e97-2302e557281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.168832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b80c2f3d-b74c-40ab-9e97-2302e557281b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:11.173186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b80c2f3d-b74c-40ab-9e97-2302e557281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:11.178249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:11.253835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 839/1688] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:11.262443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba176d-1537-448d-b15b-45318f9c5d49 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.262748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba176d-1537-448d-b15b-45318f9c5d49 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.263109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba176d-1537-448d-b15b-45318f9c5d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.298832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-68ba176d-1537-448d-b15b-45318f9c5d49 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:11.300265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-68ba176d-1537-448d-b15b-45318f9c5d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:11.310332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 850/1689] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:11.314762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c2f3d-b74c-40ab-9e97-2302e557281b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.326758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c2f3d-b74c-40ab-9e97-2302e557281b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.331449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483439-e132-4a6f-97af-680837271759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.332055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483439-e132-4a6f-97af-680837271759 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.332333 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483439-e132-4a6f-97af-680837271759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.332778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483439-e132-4a6f-97af-680837271759 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.333460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de483439-e132-4a6f-97af-680837271759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:11.333717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de483439-e132-4a6f-97af-680837271759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.335346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de483439-e132-4a6f-97af-680837271759 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.356035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 840/1690] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:04:11.361273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba176d-1537-448d-b15b-45318f9c5d49 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.371623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba176d-1537-448d-b15b-45318f9c5d49 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.376774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.377463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.377813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.378152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.378874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.381050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.385934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:11.411276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 851/1691] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:11 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:11.415547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483439-e132-4a6f-97af-680837271759 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.426639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483439-e132-4a6f-97af-680837271759 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.486662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.487310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.487592 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.487949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.488670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.490854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.515347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 841/1692] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:11 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:11.519420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.530230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb887e1-9542-4236-9cb2-fa9ee7786297 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.535240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.535985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.536327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.536695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.537472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.539187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.553167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 852/1693] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:11.554378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.567788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb01ef4-5c9b-41fb-a958-234da3dccbed admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.572561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731b105-63fc-48de-a47c-a01292167047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.573856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731b105-63fc-48de-a47c-a01292167047 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.574432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731b105-63fc-48de-a47c-a01292167047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.574728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731b105-63fc-48de-a47c-a01292167047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.575526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731b105-63fc-48de-a47c-a01292167047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.577311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731b105-63fc-48de-a47c-a01292167047 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.588283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 842/1694] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:11.592517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.602520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e2bd1-0fbf-4160-8c75-4b71676d5c26 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.607678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.608310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.608610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.608879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] PATH_INFO: `/v3/projects/6e86c1e5f9714fac82de5ee376624dc1/users/e5696d2a0bbc40d0820d27162ab7bd9a/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.625290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e86c1e5f9714fac82de5ee376624dc1, user_id=e5696d2a0bbc40d0820d27162ab7bd9a, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.628091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.641237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 853/1695] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:11 2022] PUT /identity/v3/projects/6e86c1e5f9714fac82de5ee376624dc1/users/e5696d2a0bbc40d0820d27162ab7bd9a/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:11.652547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ebe4-7443-4c8b-b30c-8513832142d9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.652944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ebe4-7443-4c8b-b30c-8513832142d9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.653320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ebe4-7443-4c8b-b30c-8513832142d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.718148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cbb9ebe4-7443-4c8b-b30c-8513832142d9 None None] MFA Rules not processed for user `e5696d2a0bbc40d0820d27162ab7bd9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:11.719410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbb9ebe4-7443-4c8b-b30c-8513832142d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:11.806718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 843/1696] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:11.811265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.881388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c8c06-147a-4df7-b743-dc7f553d3810 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.886330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.886937 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.887419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.887694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.888335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.890319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:11.919511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 854/1697] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:11 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:11.924225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ebe4-7443-4c8b-b30c-8513832142d9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:11.930793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:11.935327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ebe4-7443-4c8b-b30c-8513832142d9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:11.993470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:11.994081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:11.994361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:11.994636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:11.995328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:11.997281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.020998 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 844/1698] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:11 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:12.022318 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.033295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0910-015a-4022-bf8c-c634e3c4057d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.037893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.038516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.038795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.039073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.039733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.041316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.052396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 855/1699] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:12.056380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.066376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7dfcc-4f12-4160-a1d9-fd4b2cd62e4b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.070967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bb02a-dcfa-4aa9-8524-c3aa9f4c32a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.071594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bb02a-dcfa-4aa9-8524-c3aa9f4c32a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.071873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bb02a-dcfa-4aa9-8524-c3aa9f4c32a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.072147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bb02a-dcfa-4aa9-8524-c3aa9f4c32a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.072778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bb02a-dcfa-4aa9-8524-c3aa9f4c32a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.075152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bb02a-dcfa-4aa9-8524-c3aa9f4c32a8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.084963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 845/1700] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:12.089468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.099367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91092f65-425c-44c0-8290-a0f1f1137e3e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.104130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.104828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.105101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.105371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] PATH_INFO: `/v3/projects/986cac3d346c4e46b3ecbae55a9a1b95/users/bfad04a340094a3ba8131d89db7fe04f/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.122313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=986cac3d346c4e46b3ecbae55a9a1b95, user_id=bfad04a340094a3ba8131d89db7fe04f, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.125076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.139517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 856/1701] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:12 2022] PUT /identity/v3/projects/986cac3d346c4e46b3ecbae55a9a1b95/users/bfad04a340094a3ba8131d89db7fe04f/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:12.149963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a286c-f384-423f-803e-a444fd7e13b0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.150323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a286c-f384-423f-803e-a444fd7e13b0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.150666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a286c-f384-423f-803e-a444fd7e13b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.179116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5c4a286c-f384-423f-803e-a444fd7e13b0 None None] MFA Rules not processed for user `bfad04a340094a3ba8131d89db7fe04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:12.180408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5c4a286c-f384-423f-803e-a444fd7e13b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:12.245564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 846/1702] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:12.250421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.312834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864e678-abc5-45fb-a970-13fc1b53fbd5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.317505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.318099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.318381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.318654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.319313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.321206 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.326128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:12.352078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 857/1703] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:12 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:12.356086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a286c-f384-423f-803e-a444fd7e13b0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.367245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a286c-f384-423f-803e-a444fd7e13b0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.424805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.425410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.425684 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.425952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.426578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.428522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.449611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 847/1704] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:12 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:12.453752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.464407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e74f9-cf06-45cd-a0ae-135deed92ab1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.469361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.469933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.470213 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.470497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.471117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.472732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.482953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 858/1705] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:12.487064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.492771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:12.498738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b438a2-acc1-44d9-b3e6-64e62be3776b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.504549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.505248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.505526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.505799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] PATH_INFO: `/v3/projects/70bb8cff6e654d81ba681c92e422b27f/users/c56e03b2c46b4b89a9d32f32993b6c3f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.521192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=70bb8cff6e654d81ba681c92e422b27f, user_id=c56e03b2c46b4b89a9d32f32993b6c3f, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.523902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.536908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 848/1706] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:12 2022] PUT /identity/v3/projects/70bb8cff6e654d81ba681c92e422b27f/users/c56e03b2c46b4b89a9d32f32993b6c3f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:12.540936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.551396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef433b-9e70-4647-a661-8ec9d3cdd33a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.609095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19201c1c-5d1c-45ce-9cfb-3377eb69059e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.609713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201c1c-5d1c-45ce-9cfb-3377eb69059e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.609987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201c1c-5d1c-45ce-9cfb-3377eb69059e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.610260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201c1c-5d1c-45ce-9cfb-3377eb69059e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.610927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19201c1c-5d1c-45ce-9cfb-3377eb69059e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.612532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19201c1c-5d1c-45ce-9cfb-3377eb69059e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.623419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 859/1707] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:12.628301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.639362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d72b0c-7a88-4740-bea9-18b1b34a3fef admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.643869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.644500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.644773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.645039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] PATH_INFO: `/v3/projects/70bb8cff6e654d81ba681c92e422b27f/users/c56e03b2c46b4b89a9d32f32993b6c3f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.646897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70bb8cff6e654d81ba681c92e422b27f, user_id=c56e03b2c46b4b89a9d32f32993b6c3f, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.649631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.659505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 849/1708] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:12 2022] PUT /identity/v3/projects/70bb8cff6e654d81ba681c92e422b27f/users/c56e03b2c46b4b89a9d32f32993b6c3f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:12.669439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d7692-ba5f-4e8b-ac91-bfb8f0a0b0a2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.669725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d7692-ba5f-4e8b-ac91-bfb8f0a0b0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.670003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d7692-ba5f-4e8b-ac91-bfb8f0a0b0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.701404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-609d7692-ba5f-4e8b-ac91-bfb8f0a0b0a2 None None] MFA Rules not processed for user `c56e03b2c46b4b89a9d32f32993b6c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:12.702674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-609d7692-ba5f-4e8b-ac91-bfb8f0a0b0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:12.771730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 860/1709] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:12.783319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:12.785080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:12.842902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fac60e-7ef3-4268-91af-61021479ff00 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:12.847400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:12.848001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:12.848281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:12.848531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:12.906909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:12.908473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:12.917282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 850/1710] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:15.636422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d7692-ba5f-4e8b-ac91-bfb8f0a0b0a2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:15.650191 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d7692-ba5f-4e8b-ac91-bfb8f0a0b0a2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:15.654431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:15.655036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:15.655347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:15.655621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:15.717727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:15.723803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:15.725369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:15.735111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 861/1711] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:16.358774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:16.367143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16422188-04e4-4833-8384-a0bf252974e7 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:16.370830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0e5bd-f081-45bd-9eff-fc6ceb5f599b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:16.371461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0e5bd-f081-45bd-9eff-fc6ceb5f599b service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:16.371739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0e5bd-f081-45bd-9eff-fc6ceb5f599b service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:16.372009 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0e5bd-f081-45bd-9eff-fc6ceb5f599b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:16.380962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0e5bd-f081-45bd-9eff-fc6ceb5f599b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:16.382382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0e5bd-f081-45bd-9eff-fc6ceb5f599b service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:16.392268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 851/1712] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:20.329272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:20.394921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91195d67-6249-4d3f-b447-6e29e965a087 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:20.399390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:20.400020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:20.400298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:20.400570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] PATH_INFO: `/v3/users/e5696d2a0bbc40d0820d27162ab7bd9a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:20.401727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5696d2a0bbc40d0820d27162ab7bd9a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:20.403691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:20.451291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5696d2a0bbc40d0820d27162ab7bd9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.451935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5696d2a0bbc40d0820d27162ab7bd9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.458693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5696d2a0bbc40d0820d27162ab7bd9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.459352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5696d2a0bbc40d0820d27162ab7bd9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.470903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5696d2a0bbc40d0820d27162ab7bd9a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.484050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 862/1713] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:20 2022] DELETE /identity/v3/users/e5696d2a0bbc40d0820d27162ab7bd9a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:20.488711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:20.489250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:20.489250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:20.508582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:20.509764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:20.569273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 852/1714] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:20.582100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:20.583628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:20.641656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccfd8-5356-4350-9ced-0f13c4704fc3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:20.653843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d4ea8-d022-40a9-92a4-e8e22d6a9a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:20.654483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d4ea8-d022-40a9-92a4-e8e22d6a9a79 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:20.654764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d4ea8-d022-40a9-92a4-e8e22d6a9a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:20.655037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d4ea8-d022-40a9-92a4-e8e22d6a9a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:20.666271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d4ea8-d022-40a9-92a4-e8e22d6a9a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:20.667709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d4ea8-d022-40a9-92a4-e8e22d6a9a79 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:20.677813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 863/1715] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:20.825600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:20.827758 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:20.841621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fdede-9030-49ff-8703-02bb2bc87cad None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:20.846238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:20.846860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:20.847352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:20.847620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] PATH_INFO: `/v3/users/af0d79ae883d43a59af2508ea62000e9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:20.848714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0d79ae883d43a59af2508ea62000e9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:20.850945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:20.852735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:20.906242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0d79ae883d43a59af2508ea62000e9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.906953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0d79ae883d43a59af2508ea62000e9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.916268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0d79ae883d43a59af2508ea62000e9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.924868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0d79ae883d43a59af2508ea62000e9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.925584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0d79ae883d43a59af2508ea62000e9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:20.945056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 853/1716] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:20 2022] DELETE /identity/v3/users/af0d79ae883d43a59af2508ea62000e9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:20.952409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:20.954518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:20.954518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:20.972035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:20.973521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:20.977458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:20.979133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:21.002579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:21.057159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa377bb6-ced7-4984-8030-6638f4192f88 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:21.063011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:21.063715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:21.064002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:21.064254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] PATH_INFO: `/v3/projects/6e86c1e5f9714fac82de5ee376624dc1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:21.065358 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e86c1e5f9714fac82de5ee376624dc1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:21.067356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:21.077082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 864/1717] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:21.087591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:21.089153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:21.127835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e86c1e5f9714fac82de5ee376624dc1', 'request_id': 'req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.134647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e86c1e5f9714fac82de5ee376624dc1', 'request_id': 'req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.142774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e86c1e5f9714fac82de5ee376624dc1', 'request_id': 'req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.162178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] The token cache is being invalidate because project 6e86c1e5f9714fac82de5ee376624dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:21.162591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.163603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.163720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6348a-e66b-4168-9587-c598dba34434 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:21.165317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 854/1718] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:20 2022] DELETE /identity/v3/projects/6e86c1e5f9714fac82de5ee376624dc1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:21.169796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:21.171179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:21.188696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:21.223765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:21.224426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:21.224704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:21.225277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:21.227431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:21.298737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c50f2d-a1fa-4527-8f53-5e49d4fc828f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:21.305100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:21.306073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:21.306495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:21.307146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] PATH_INFO: `/v3/users/bfad04a340094a3ba8131d89db7fe04f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:21.309206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfad04a340094a3ba8131d89db7fe04f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:21.312509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:21.336341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:21.339500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:21.356991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 865/1719] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:21.395031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfad04a340094a3ba8131d89db7fe04f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.398224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfad04a340094a3ba8131d89db7fe04f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.420250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfad04a340094a3ba8131d89db7fe04f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.429381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfad04a340094a3ba8131d89db7fe04f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.430258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfad04a340094a3ba8131d89db7fe04f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.459661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 855/1720] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:21 2022] DELETE /identity/v3/users/bfad04a340094a3ba8131d89db7fe04f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:21.760804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:21.762932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:21.801861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b552559-e935-4116-a09f-401609c3fa81 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:21.802622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:21.815147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:21.839159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:21.887168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:21.887827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:21.888101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:21.888367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] PATH_INFO: `/v3/projects/4e0680c2e78b46a8b946d68f0a26f5d4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:21.889542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0680c2e78b46a8b946d68f0a26f5d4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:21.891915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:21.895488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04f6ff-fccf-49d2-a26c-c5dce251ea43 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:21.900361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:21.901044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:21.901872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:21.902093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] PATH_INFO: `/v3/projects/986cac3d346c4e46b3ecbae55a9a1b95` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:21.903494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] RBAC: Authorizing `identity:delete_project(project_id=986cac3d346c4e46b3ecbae55a9a1b95)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:21.905411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:21.940017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:21.941457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0680c2e78b46a8b946d68f0a26f5d4', 'request_id': 'req-1b8c0597-6864-4b7d-86ff-b50419883b45'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.948434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0680c2e78b46a8b946d68f0a26f5d4', 'request_id': 'req-1b8c0597-6864-4b7d-86ff-b50419883b45'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.955413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0680c2e78b46a8b946d68f0a26f5d4', 'request_id': 'req-1b8c0597-6864-4b7d-86ff-b50419883b45'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.955553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986cac3d346c4e46b3ecbae55a9a1b95', 'request_id': 'req-ce438b95-3bff-4a73-816a-097d4213462f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.962868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986cac3d346c4e46b3ecbae55a9a1b95', 'request_id': 'req-ce438b95-3bff-4a73-816a-097d4213462f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.969856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] The token cache is being invalidate because project 4e0680c2e78b46a8b946d68f0a26f5d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:21.970223 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.970852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.971200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986cac3d346c4e46b3ecbae55a9a1b95', 'request_id': 'req-ce438b95-3bff-4a73-816a-097d4213462f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.972516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 866/1721] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:21 2022] DELETE /identity/v3/projects/4e0680c2e78b46a8b946d68f0a26f5d4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:21.976449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:21.978033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:21.983311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] The token cache is being invalidate because project 986cac3d346c4e46b3ecbae55a9a1b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:21.983602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.984251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:21.985973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 856/1722] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:21 2022] DELETE /identity/v3/projects/986cac3d346c4e46b3ecbae55a9a1b95 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:21.989485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:21.991029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:22.044612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c0597-6864-4b7d-86ff-b50419883b45 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:22.064865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438b95-3bff-4a73-816a-097d4213462f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:22.070509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:22.071148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:22.071452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:22.071730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] PATH_INFO: `/v3/users/c56e03b2c46b4b89a9d32f32993b6c3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:22.073363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56e03b2c46b4b89a9d32f32993b6c3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:22.075696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:22.103844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:22.104553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:22.104830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:22.105104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] PATH_INFO: `/v3/users/f82e6f3709c0452d8e32a897a35092ad` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:22.106246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82e6f3709c0452d8e32a897a35092ad)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:22.108383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:22.110007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:22.124843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56e03b2c46b4b89a9d32f32993b6c3f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.125550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56e03b2c46b4b89a9d32f32993b6c3f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.135030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56e03b2c46b4b89a9d32f32993b6c3f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.144110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56e03b2c46b4b89a9d32f32993b6c3f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.144762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56e03b2c46b4b89a9d32f32993b6c3f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.163876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 857/1723] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:21 2022] DELETE /identity/v3/users/c56e03b2c46b4b89a9d32f32993b6c3f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:22.173071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82e6f3709c0452d8e32a897a35092ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.173733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82e6f3709c0452d8e32a897a35092ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.181518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82e6f3709c0452d8e32a897a35092ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.182900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82e6f3709c0452d8e32a897a35092ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.201118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82e6f3709c0452d8e32a897a35092ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.216463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 867/1724] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:21 2022] DELETE /identity/v3/users/f82e6f3709c0452d8e32a897a35092ad => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:22.524117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:22.527595 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:22.606181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:22.607022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:22.609321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:22.652274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f760f54-e45a-4516-ab22-5e6e3f42859f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:22.720840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d133-65e0-488b-ad19-a6e1eb81c9a9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:22.727191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:22.729226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:22.729708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:22.730168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] PATH_INFO: `/v3/projects/70bb8cff6e654d81ba681c92e422b27f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:22.731570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70bb8cff6e654d81ba681c92e422b27f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:22.733690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:22.744512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:22.745201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:22.745484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:22.745882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] PATH_INFO: `/v3/projects/fb3f8d8ac03e45d0976048d68539bd82` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:22.747041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3f8d8ac03e45d0976048d68539bd82)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:22.749079 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:22.813963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70bb8cff6e654d81ba681c92e422b27f', 'request_id': 'req-96ba16b4-71e8-4fa6-a904-58d0ce342959'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.826003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3f8d8ac03e45d0976048d68539bd82', 'request_id': 'req-2d3a2cc8-80ce-48c0-828f-c18faaa88616'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.826497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70bb8cff6e654d81ba681c92e422b27f', 'request_id': 'req-96ba16b4-71e8-4fa6-a904-58d0ce342959'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.833776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3f8d8ac03e45d0976048d68539bd82', 'request_id': 'req-2d3a2cc8-80ce-48c0-828f-c18faaa88616'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.837630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70bb8cff6e654d81ba681c92e422b27f', 'request_id': 'req-96ba16b4-71e8-4fa6-a904-58d0ce342959'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.853410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3f8d8ac03e45d0976048d68539bd82', 'request_id': 'req-2d3a2cc8-80ce-48c0-828f-c18faaa88616'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.856092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] The token cache is being invalidate because project 70bb8cff6e654d81ba681c92e422b27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:22.857526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.858132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.859909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 858/1725] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:22 2022] DELETE /identity/v3/projects/70bb8cff6e654d81ba681c92e422b27f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:22.860744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:22.865078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:22.871102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] The token cache is being invalidate because project fb3f8d8ac03e45d0976048d68539bd82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:22.871856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.872148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d3a2cc8-80ce-48c0-828f-c18faaa88616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:22.875418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 868/1726] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:22 2022] DELETE /identity/v3/projects/fb3f8d8ac03e45d0976048d68539bd82 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:22.875418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3e67-bd7c-4d1e-8242-9ba9290d943d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:22.875639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3e67-bd7c-4d1e-8242-9ba9290d943d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:22.875804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3e67-bd7c-4d1e-8242-9ba9290d943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:22.904783 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-71fc3e67-bd7c-4d1e-8242-9ba9290d943d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:22.904783 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-71fc3e67-bd7c-4d1e-8242-9ba9290d943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:22.911328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:22.944291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba16b4-71e8-4fa6-a904-58d0ce342959 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:22.967473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:22.975477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 869/1727] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:22.976915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c847-e936-4d2f-b0b8-c21f15d31fd8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:22.977209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c847-e936-4d2f-b0b8-c21f15d31fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:22.977484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c847-e936-4d2f-b0b8-c21f15d31fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.001926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7243c847-e936-4d2f-b0b8-c21f15d31fd8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:23.003145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7243c847-e936-4d2f-b0b8-c21f15d31fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.012602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 870/1728] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:23.014163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003b4d2-6679-494d-a4ee-869bdeaff2fb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.014670 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003b4d2-6679-494d-a4ee-869bdeaff2fb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.015112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003b4d2-6679-494d-a4ee-869bdeaff2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.028415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.031258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.031258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.031258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] PATH_INFO: `/v3/users/a3a10f7f14e24e49adb84359a5f309b1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.031258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a10f7f14e24e49adb84359a5f309b1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.037484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.038953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5003b4d2-6679-494d-a4ee-869bdeaff2fb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:23.040918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5003b4d2-6679-494d-a4ee-869bdeaff2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.053029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 871/1729] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:23.054395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100190fb-466b-48f5-81f9-87b01f57acba None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.054759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100190fb-466b-48f5-81f9-87b01f57acba None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.055036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100190fb-466b-48f5-81f9-87b01f57acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.074685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-100190fb-466b-48f5-81f9-87b01f57acba None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:23.075926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-100190fb-466b-48f5-81f9-87b01f57acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.084007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 872/1730] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:23.087951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100190fb-466b-48f5-81f9-87b01f57acba None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.093294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a10f7f14e24e49adb84359a5f309b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:23.094312 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a10f7f14e24e49adb84359a5f309b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:23.098000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100190fb-466b-48f5-81f9-87b01f57acba None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.104185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.105334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.105595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.105861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.106502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:23.106808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.108356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a10f7f14e24e49adb84359a5f309b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:23.108484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.116868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a10f7f14e24e49adb84359a5f309b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:23.117611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a10f7f14e24e49adb84359a5f309b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:23.123115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 873/1731] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:23.127255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.129061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.140000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 859/1732] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:22 2022] DELETE /identity/v3/users/a3a10f7f14e24e49adb84359a5f309b1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:23.144175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.145801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.145918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685773f-9a3a-4418-9a65-bcfed874d261 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.221906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371f361-5ec8-4ec9-aaa0-5711e6f527cc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.227552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.228442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.228813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.229255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.229976 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.231032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be1525-2893-4c68-a346-fad919f9d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.231032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1525-2893-4c68-a346-fad919f9d2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.231381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1525-2893-4c68-a346-fad919f9d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.231715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1525-2893-4c68-a346-fad919f9d2d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.232563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be1525-2893-4c68-a346-fad919f9d2d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:23.232724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.232906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be1525-2893-4c68-a346-fad919f9d2d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.235129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be1525-2893-4c68-a346-fad919f9d2d4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.252150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 874/1733] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:23.253501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caebc-edab-4116-9c9e-df798e1e3da5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.253856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caebc-edab-4116-9c9e-df798e1e3da5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.254215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caebc-edab-4116-9c9e-df798e1e3da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.276824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7a0caebc-edab-4116-9c9e-df798e1e3da5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:23.278165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7a0caebc-edab-4116-9c9e-df798e1e3da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.286472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 860/1734] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:23.287010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 875/1735] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:23.287650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.291122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0caebc-edab-4116-9c9e-df798e1e3da5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.301678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0caebc-edab-4116-9c9e-df798e1e3da5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.336501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:23.367673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.368281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.368556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.368825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.369447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.371470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.378633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98b71-dada-45b1-8f0b-e68269f76739 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.382912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:23.385862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.386772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.387051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.387363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.388019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.390341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.404894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 876/1736] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:23.405741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.409083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.424825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 861/1737] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:23.425929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.477055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47191270-0eaa-4d77-b094-231f08e96c5b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.497410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258713d1-066f-4a28-8e5d-c61032c955fb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.502030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.503026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.503026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.503260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.503828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.505438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.515997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 862/1738] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:23.519721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.529607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148603c-804d-4c2e-abf6-691a579711c4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.534971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.535670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.535947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.536243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.536978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.538765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.544596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.545285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.545800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.546082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.556754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.559407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.567106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 863/1739] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:23.567106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.571053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 877/1740] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:23.571903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.576883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd74c5-edf7-4598-88ee-1c356f8360e4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.582666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927c9a1-b49d-42ab-bd64-8b74d1f420cc service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.588020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.588683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.589639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.589920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.591262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.593408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.596822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.597442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.597723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.597995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.598633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.600393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.609242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 864/1741] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:23.613239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.613893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 878/1742] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:23.617463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.625776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b1633-f017-41c2-8dae-46ac7afd0064 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.628823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc438c9-219a-4624-98c2-e15c6c9a4b1b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.632375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e52817-7711-4477-9fda-b4e9b8e32943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.633461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52817-7711-4477-9fda-b4e9b8e32943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.633954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52817-7711-4477-9fda-b4e9b8e32943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.634371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.634755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52817-7711-4477-9fda-b4e9b8e32943 admin admin] PATH_INFO: `/v3/projects/d271a44fc1314e67b6af3fe514d164d9/users/fd4f27a9b2b642bda08cb3ab05147e96/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.635128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.635680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.636024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.636964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.638716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.650801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 879/1743] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:23.654631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.655988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e52817-7711-4477-9fda-b4e9b8e32943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d271a44fc1314e67b6af3fe514d164d9, user_id=fd4f27a9b2b642bda08cb3ab05147e96, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.661256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e52817-7711-4477-9fda-b4e9b8e32943 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.672869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4115-4b0a-4ea1-bb0e-8d73fc3d2ba2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.684350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b7cb2-8e02-40ca-9294-dc064f9412b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.685058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b7cb2-8e02-40ca-9294-dc064f9412b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.685315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b7cb2-8e02-40ca-9294-dc064f9412b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.685590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b7cb2-8e02-40ca-9294-dc064f9412b2 admin admin] PATH_INFO: `/v3/projects/e8a98a4bedab4f87b69b14e54d75950d/users/036d33e7289a409c9f9202eb2cbd9a59/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.690841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 865/1744] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:23 2022] PUT /identity/v3/projects/d271a44fc1314e67b6af3fe514d164d9/users/fd4f27a9b2b642bda08cb3ab05147e96/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:23.702798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.703085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.703413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.719752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b7cb2-8e02-40ca-9294-dc064f9412b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a98a4bedab4f87b69b14e54d75950d, user_id=036d33e7289a409c9f9202eb2cbd9a59, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:23.724205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b7cb2-8e02-40ca-9294-dc064f9412b2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:23.740678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 880/1745] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:23 2022] PUT /identity/v3/projects/e8a98a4bedab4f87b69b14e54d75950d/users/036d33e7289a409c9f9202eb2cbd9a59/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:23.744872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] MFA Rules not processed for user `fd4f27a9b2b642bda08cb3ab05147e96`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:23.746064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.751610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022a4f3-9f39-4502-abbd-bf5a09420447 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.752092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022a4f3-9f39-4502-abbd-bf5a09420447 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.752472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022a4f3-9f39-4502-abbd-bf5a09420447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:23.816480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f022a4f3-9f39-4502-abbd-bf5a09420447 None None] MFA Rules not processed for user `036d33e7289a409c9f9202eb2cbd9a59`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:23.817739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f022a4f3-9f39-4502-abbd-bf5a09420447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.821154 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:23.893992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 866/1746] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:23.910792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.910792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:23.934105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:23.944222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 881/1747] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:23.949515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022a4f3-9f39-4502-abbd-bf5a09420447 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:23.991041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24d735-4586-48d7-a06f-aa9c973ff1da None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:23.995624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:23.996465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:23.996580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:23.996768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.007076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.008540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.020533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 867/1748] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:24.029195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022a4f3-9f39-4502-abbd-bf5a09420447 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.033884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.034508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.034786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.035053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.035836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.042964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.077960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 882/1749] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:23 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:24.079339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.090379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f41a2-31b8-49c0-875c-63ac3031c891 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.169407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.169959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.170238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.170508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.171144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.173366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.212698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 868/1750] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:24.217766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.234120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5b818-a21e-4110-87ba-710660d6bf27 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.240897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.242009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.242342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.242662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.243624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.246353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.270521 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 883/1751] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:24.271826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.284451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c78a3-1d4e-4937-8dfe-8fc72fa6f51f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.290648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71400fdb-7237-4a06-a47f-346906846701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.291319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71400fdb-7237-4a06-a47f-346906846701 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.292117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71400fdb-7237-4a06-a47f-346906846701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.292391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71400fdb-7237-4a06-a47f-346906846701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.293042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71400fdb-7237-4a06-a47f-346906846701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.295431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71400fdb-7237-4a06-a47f-346906846701 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.315263 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 869/1752] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:24.317411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.332646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732846c-a277-4f97-8107-70b802bf62c0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.344021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.344710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.345095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.345461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] PATH_INFO: `/v3/projects/413eff6ee1534001b74994d9e5e3db06/users/37d4df7ea1494c2081bc2606d34c5863/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.374310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] RBAC: Authorizing `identity:create_grant(project_id=413eff6ee1534001b74994d9e5e3db06, user_id=37d4df7ea1494c2081bc2606d34c5863, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.377288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.405413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 884/1753] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:24 2022] PUT /identity/v3/projects/413eff6ee1534001b74994d9e5e3db06/users/37d4df7ea1494c2081bc2606d34c5863/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:24.415911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eacc9-db7e-4359-9cad-c9e43ddbc3a3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.418498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eacc9-db7e-4359-9cad-c9e43ddbc3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.419373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eacc9-db7e-4359-9cad-c9e43ddbc3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.441269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.445841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:24.467583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bc1eacc9-db7e-4359-9cad-c9e43ddbc3a3 None None] MFA Rules not processed for user `37d4df7ea1494c2081bc2606d34c5863`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:24.467583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bc1eacc9-db7e-4359-9cad-c9e43ddbc3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:24.592784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184b515-53b7-411a-80ef-db60061b363a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.598556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.599427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.599865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.600264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] PATH_INFO: `/v3/projects/2f61d3c56dbe430885109d5b42b0c272` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.601536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f61d3c56dbe430885109d5b42b0c272)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.604459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.625132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:24.637667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 870/1754] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:24.642800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eacc9-db7e-4359-9cad-c9e43ddbc3a3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.665589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eacc9-db7e-4359-9cad-c9e43ddbc3a3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.666323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:24.668355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f61d3c56dbe430885109d5b42b0c272', 'request_id': 'req-598d766f-d79a-443a-8a72-a572242d480a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:24.680045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f61d3c56dbe430885109d5b42b0c272', 'request_id': 'req-598d766f-d79a-443a-8a72-a572242d480a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:24.687508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f61d3c56dbe430885109d5b42b0c272', 'request_id': 'req-598d766f-d79a-443a-8a72-a572242d480a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:24.705728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] The token cache is being invalidate because project 2f61d3c56dbe430885109d5b42b0c272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:24.706080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:24.706677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-598d766f-d79a-443a-8a72-a572242d480a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:24.710082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 885/1755] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:24 2022] DELETE /identity/v3/projects/2f61d3c56dbe430885109d5b42b0c272 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:24.715026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db174ca0-f92c-4e98-911e-222dfb3c08e8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.715506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db174ca0-f92c-4e98-911e-222dfb3c08e8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.717301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db174ca0-f92c-4e98-911e-222dfb3c08e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.743864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-db174ca0-f92c-4e98-911e-222dfb3c08e8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:24.745965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db174ca0-f92c-4e98-911e-222dfb3c08e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:24.748426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.749102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.749421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.749890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.750659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.752953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.761386 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 886/1756] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:24.763325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7a27-6447-4642-9aef-a1e66ab49ae3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.763555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7a27-6447-4642-9aef-a1e66ab49ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.764144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7a27-6447-4642-9aef-a1e66ab49ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.785638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-75ea7a27-6447-4642-9aef-a1e66ab49ae3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:24.786969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-75ea7a27-6447-4642-9aef-a1e66ab49ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:24.796720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 871/1757] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:24.799307 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 887/1758] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:24.801452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7a27-6447-4642-9aef-a1e66ab49ae3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.801631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.803700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:24.813902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7a27-6447-4642-9aef-a1e66ab49ae3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.881275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.881546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06851415-5b97-445d-a35f-a53b489b152a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.882053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.882453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.882919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.883708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:24.884117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.886535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.887652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.888576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.888852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.889123 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.889997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.892867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.928129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 888/1759] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:24.929007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.934158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 872/1760] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:24.938694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:24.941788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e1051-b136-4d23-921f-67c2074188e3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.947285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.948502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.948975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.949603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.950347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.952367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.958514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd208-8636-4f75-8044-f2909fa4506a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:24.965977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:24.966869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:24.967195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:24.967846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:24.975523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:24.977732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:24.994881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 873/1762] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:24.996100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.004045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 889/1762] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:25.008707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f544c-f8a4-40a9-932b-483e8e6ef039 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.018234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.119696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.120412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.120819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.121265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.122031 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.124164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.136384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:25.150509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429896-ed19-471c-abc8-d6ead94e6633 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.157415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.158169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.158443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.158721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] PATH_INFO: `/v3/projects/100c0d9e03a84185aa72a50a8f4ff199/users/295502558219481b87f2fa74e9a53edc/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.159876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 874/1763] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:24 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:25.164272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.176685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7304888-de22-485a-a5a5-7a4e432bc2c7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.178417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=100c0d9e03a84185aa72a50a8f4ff199, user_id=295502558219481b87f2fa74e9a53edc, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.181477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.181699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.182088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.183132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.183559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.184289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.186152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.204322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 890/1764] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:25 2022] PUT /identity/v3/projects/100c0d9e03a84185aa72a50a8f4ff199/users/295502558219481b87f2fa74e9a53edc/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:25.209271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.214532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 875/1765] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:25.215974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.227070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1d1dd-4978-4dcc-9b5a-994e06d5f63a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.239882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:25.320757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66103e9a-55d1-47be-bc2b-8471a513f20a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.324572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.325717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.326275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.327280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.328235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.330727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.338877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.340377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.340377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.340661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.341279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.343710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.348194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 876/1766] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:25.357374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.364820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 891/1767] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:25.367388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.377413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7d11d-78af-4114-b810-4d35fe1d3935 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.385087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526df5ba-5274-49a7-b9af-cc78451ebdaa admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.392203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479f830-4414-431d-855b-9a410c52b325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.393202 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f830-4414-431d-855b-9a410c52b325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.393669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f830-4414-431d-855b-9a410c52b325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.394300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f830-4414-431d-855b-9a410c52b325 admin admin] PATH_INFO: `/v3/projects/100c0d9e03a84185aa72a50a8f4ff199/users/295502558219481b87f2fa74e9a53edc/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.397190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479f830-4414-431d-855b-9a410c52b325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100c0d9e03a84185aa72a50a8f4ff199, user_id=295502558219481b87f2fa74e9a53edc, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.400690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ef80c-8435-45ef-bbb2-ce6ff2d91260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.401186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479f830-4414-431d-855b-9a410c52b325 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.402417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef80c-8435-45ef-bbb2-ce6ff2d91260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.402618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef80c-8435-45ef-bbb2-ce6ff2d91260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.403558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef80c-8435-45ef-bbb2-ce6ff2d91260 admin admin] PATH_INFO: `/v3/projects/170a0b943fd847c3a46eb4c5e1df125c/users/b705a02c504e46188c903bf2ce4952de/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.417743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 892/1768] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:25 2022] PUT /identity/v3/projects/100c0d9e03a84185aa72a50a8f4ff199/users/295502558219481b87f2fa74e9a53edc/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:25.430855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.431140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.431626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.436447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ef80c-8435-45ef-bbb2-ce6ff2d91260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170a0b943fd847c3a46eb4c5e1df125c, user_id=b705a02c504e46188c903bf2ce4952de, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.439313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ef80c-8435-45ef-bbb2-ce6ff2d91260 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.479834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] MFA Rules not processed for user `295502558219481b87f2fa74e9a53edc`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:25.480564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 877/1769] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:25 2022] PUT /identity/v3/projects/170a0b943fd847c3a46eb4c5e1df125c/users/b705a02c504e46188c903bf2ce4952de/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:25.481255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:25.482060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca9218-955a-4146-b23a-7d53659868b3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.482431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca9218-955a-4146-b23a-7d53659868b3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.482625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca9218-955a-4146-b23a-7d53659868b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.526891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d8ca9218-955a-4146-b23a-7d53659868b3 None None] MFA Rules not processed for user `b705a02c504e46188c903bf2ce4952de`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:25.528283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d8ca9218-955a-4146-b23a-7d53659868b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:25.653471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 878/1770] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:25.654689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca9218-955a-4146-b23a-7d53659868b3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.662021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 893/1771] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:25.673141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.684476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:25.795075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:25.825912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bddc4b-3b27-4494-8c09-c6acc5244a13 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.830471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca9218-955a-4146-b23a-7d53659868b3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:25.836094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.836723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.837002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.837273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.840931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:25.841556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:25.841828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:25.842098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:25.842735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.845042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.887244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:25.888105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:25.922282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 894/1772] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:25.932039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 879/1773] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:25 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:25.937209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:25.955786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cea4c3-e04f-4169-bd3b-cce883e981c3 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.017691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:26.106281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.106847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.107124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.107419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.108051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.111494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.153587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 895/1774] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:25 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:26.164327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.180287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc803f6-92d9-4aeb-ab56-91bcc36c7c48 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.197638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.198516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.208190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.208568 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.209328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.211670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.258906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.273357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 880/1775] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:26.273558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68763346-f991-4b29-a0a9-27b6cebe4b69 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.289556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f20be-c5f6-4d85-83a5-7beff624fbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.291960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f20be-c5f6-4d85-83a5-7beff624fbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.292311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f20be-c5f6-4d85-83a5-7beff624fbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.292651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f20be-c5f6-4d85-83a5-7beff624fbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.298958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f20be-c5f6-4d85-83a5-7beff624fbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.300737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f20be-c5f6-4d85-83a5-7beff624fbc6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.324965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 896/1776] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:26.327041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.348049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146b8d6-5ce8-4c8a-b7d1-7145df3a1c62 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.355093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.362414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.371153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.374039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] PATH_INFO: `/v3/projects/a73521654f4241e1a33470db823c5809/users/9d48c962ac1243ae9c645785febda074/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.409200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73521654f4241e1a33470db823c5809, user_id=9d48c962ac1243ae9c645785febda074, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.412288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.459559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 881/1777] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:26 2022] PUT /identity/v3/projects/a73521654f4241e1a33470db823c5809/users/9d48c962ac1243ae9c645785febda074/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:26.469176 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995328ae-ddee-4193-bdbb-94841d631489 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.469730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995328ae-ddee-4193-bdbb-94841d631489 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.470358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995328ae-ddee-4193-bdbb-94841d631489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.535747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-995328ae-ddee-4193-bdbb-94841d631489 None None] MFA Rules not processed for user `9d48c962ac1243ae9c645785febda074`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:26.537383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-995328ae-ddee-4193-bdbb-94841d631489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:26.615259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 897/1778] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:26.619473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.688434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8661c-95ff-460e-b3b7-7915cc25ac32 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.695869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.696480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.696757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.697024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.697656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.700559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.729426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 882/1779] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:26 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:26.734014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995328ae-ddee-4193-bdbb-94841d631489 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.754906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995328ae-ddee-4193-bdbb-94841d631489 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.766340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.768809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:26.832843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.833176 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.833456 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.833730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.834526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.858472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031a456-8fc3-49ab-a98f-b752f24962c1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.862411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.863127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.863508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.863858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.865729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:26.866346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.903835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:26.910104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 898/1780] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:26 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:26.910777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.922998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6588-553e-4f8f-8ff5-0f8a84bcb6bb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.929435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.930197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.930478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.930753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.931489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.933413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.941368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.943524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:26.948508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 899/1781] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:26.952820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:26.967275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd32001-2f57-4c13-8600-b0d87abf3e17 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:26.970137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 883/1782] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:26.970323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:26.971012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:26.971392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:26.971672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] PATH_INFO: `/v3/projects/99949ef1619a4ca69f45022bde486919/users/a658a884fc29403a8cbeb93b7e47586f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:26.987274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:26.991607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99949ef1619a4ca69f45022bde486919, user_id=a658a884fc29403a8cbeb93b7e47586f, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:26.995462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:27.021075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 900/1783] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:26 2022] PUT /identity/v3/projects/99949ef1619a4ca69f45022bde486919/users/a658a884fc29403a8cbeb93b7e47586f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:27.027098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:27.051768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb22d3a-6738-481e-b302-e2aa0d635f98 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:27.143245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40b785-0fce-41f2-9400-5a6a550b8697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:27.143893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40b785-0fce-41f2-9400-5a6a550b8697 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:27.144175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40b785-0fce-41f2-9400-5a6a550b8697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:27.144446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40b785-0fce-41f2-9400-5a6a550b8697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:27.145361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40b785-0fce-41f2-9400-5a6a550b8697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:27.147467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40b785-0fce-41f2-9400-5a6a550b8697 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:27.159271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 884/1784] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:27.163810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:27.196981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fda2b5-126c-4d65-b3b7-51491899afb4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:27.218049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:27.219111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:27.219111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:27.219362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] PATH_INFO: `/v3/projects/99949ef1619a4ca69f45022bde486919/users/a658a884fc29403a8cbeb93b7e47586f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:27.227820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99949ef1619a4ca69f45022bde486919, user_id=a658a884fc29403a8cbeb93b7e47586f, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:27.231423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:27.250497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 901/1785] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:27 2022] PUT /identity/v3/projects/99949ef1619a4ca69f45022bde486919/users/a658a884fc29403a8cbeb93b7e47586f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:27.261474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:27.261842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:27.262183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:27.340232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] MFA Rules not processed for user `a658a884fc29403a8cbeb93b7e47586f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:27.341468 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:27.552183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 885/1786] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:27.586797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:27.670122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7d25b-a2ef-471a-a840-d940e824649b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:27.675540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:27.676240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:27.676595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:27.677162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:27.762491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:27.764765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:27.776225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 902/1787] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:27.814570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:27.816415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:27.912500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:27.940291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3b1a8-aaa1-4d8f-a32f-60d7ff237285 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:27.959028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:27.959844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:27.961371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:27.963566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] PATH_INFO: `/v3/users/8d144842b98e477aaef275278ad8db7e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:27.965011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d144842b98e477aaef275278ad8db7e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:27.967623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.031623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d144842b98e477aaef275278ad8db7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.032571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d144842b98e477aaef275278ad8db7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.041110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d144842b98e477aaef275278ad8db7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.051450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d144842b98e477aaef275278ad8db7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.052222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a9e829a-abe6-45f0-81fe-6fc28e08fd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d144842b98e477aaef275278ad8db7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.072998 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 886/1788] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:27 2022] DELETE /identity/v3/users/8d144842b98e477aaef275278ad8db7e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:28.269882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.283315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e4e03-d70f-4729-a8a1-449a3196772f service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:28.340751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:28.341414 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.341714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.341988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] PATH_INFO: `/v3/projects/b07dcfcb56324cc3a64ade8b763ae573` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.343089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07dcfcb56324cc3a64ade8b763ae573)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:28.345109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.383412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:28.394563 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07dcfcb56324cc3a64ade8b763ae573', 'request_id': 'req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.400280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07dcfcb56324cc3a64ade8b763ae573', 'request_id': 'req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.407565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07dcfcb56324cc3a64ade8b763ae573', 'request_id': 'req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.421917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] The token cache is being invalidate because project b07dcfcb56324cc3a64ade8b763ae573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:28.422355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.423097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec2c8f61-a0ca-4c70-a6a8-75cbf58981ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:28.424864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 903/1789] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:28 2022] DELETE /identity/v3/projects/b07dcfcb56324cc3a64ade8b763ae573 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:28.430060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65910c-151a-4697-b76a-d144052156ef None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.430417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65910c-151a-4697-b76a-d144052156ef None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.430761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65910c-151a-4697-b76a-d144052156ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.451351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bf65910c-151a-4697-b76a-d144052156ef None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:28.452733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bf65910c-151a-4697-b76a-d144052156ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:28.518105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 887/1790] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:28.526715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc10e7-dad3-43ea-a76e-e51471156d7e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.527076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc10e7-dad3-43ea-a76e-e51471156d7e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.527449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc10e7-dad3-43ea-a76e-e51471156d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.548864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-adbc10e7-dad3-43ea-a76e-e51471156d7e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:28.550177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-adbc10e7-dad3-43ea-a76e-e51471156d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:28.558615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 904/1791] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:28.562692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65910c-151a-4697-b76a-d144052156ef None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.573283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65910c-151a-4697-b76a-d144052156ef None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:28.578286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:28.578883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.579159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.579490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.580126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:28.580428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:28.582088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.598928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 888/1792] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:28.600125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc10e7-dad3-43ea-a76e-e51471156d7e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.609173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc10e7-dad3-43ea-a76e-e51471156d7e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:28.613825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:28.614408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.614679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.614943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.615591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:28.617491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.645136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 905/1793] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:28 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:28.649059 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.660286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75480f-38e7-4e63-9ada-e06962e98f9e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:28.723995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:28.731497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:28.732313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.732808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.733611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.734419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:28.736507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.758428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 889/1794] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:28 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:28.763218 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.774540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ea57a-f9e9-4ef9-a7e9-0eaaaaaef40f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:28.779827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:28.780450 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.780726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.780993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.781646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:28.783235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.796061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 906/1795] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:28.798239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.808854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e69d2-84d2-48d8-b62d-7ae47c5e33fe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:28.818063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96087e-7e28-46e1-8a07-50fedce8af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:28.818714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96087e-7e28-46e1-8a07-50fedce8af71 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.818987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96087e-7e28-46e1-8a07-50fedce8af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.819292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96087e-7e28-46e1-8a07-50fedce8af71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.819929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96087e-7e28-46e1-8a07-50fedce8af71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:28.821510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96087e-7e28-46e1-8a07-50fedce8af71 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.832997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 890/1796] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:28.837248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.848399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ba7d-a363-4ce1-81b1-1efad849337e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:28.853060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:28.853726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.854102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.854376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] PATH_INFO: `/v3/projects/721190b4f16142749dda7253941be2c3/users/8d404b2974f64aa8a6c32adfc3f7b0bc/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.868622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] RBAC: Authorizing `identity:create_grant(project_id=721190b4f16142749dda7253941be2c3, user_id=8d404b2974f64aa8a6c32adfc3f7b0bc, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:28.871482 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:28.884700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 907/1797] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:28 2022] PUT /identity/v3/projects/721190b4f16142749dda7253941be2c3/users/8d404b2974f64aa8a6c32adfc3f7b0bc/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:28.890309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:28.890596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:28.890883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:28.910802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:28.912015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:28.973406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 891/1798] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:28.982936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:28.984650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:29.043244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132c7c5-10ed-45e2-9bf6-775ba883259d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:29.047237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:29.047841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:29.048116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:29.048384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:29.057890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:29.059199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:29.069136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 908/1799] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:29.471002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:29.473067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:29.560498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4634e2-eb3f-42e9-b4b2-6bd87ef191fd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:29.567309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:29.568279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:29.568658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:29.568992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:29.574613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:29.655323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:29.657029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:29.670313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 892/1800] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:30.639926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:30.641431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:30.656034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cac56-356f-44ef-9e33-cf928482ae1d service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:30.660948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929ae03-0998-48c0-9b0d-dc7fcaa4c26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:30.661994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929ae03-0998-48c0-9b0d-dc7fcaa4c26e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:30.662447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929ae03-0998-48c0-9b0d-dc7fcaa4c26e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:30.663162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929ae03-0998-48c0-9b0d-dc7fcaa4c26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:30.665316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b929ae03-0998-48c0-9b0d-dc7fcaa4c26e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:30.759505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:30.763117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929ae03-0998-48c0-9b0d-dc7fcaa4c26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:30.765234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929ae03-0998-48c0-9b0d-dc7fcaa4c26e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:30.788378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 909/1801] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:31.811829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:31.829042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520274d-8bea-41a1-9010-35965da85aa1 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:31.832925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:31.833687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:31.833973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:31.834255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:31.840680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:31.899607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:31.976795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:31.978591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:31.991851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 893/1802] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:34.669594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:34.670123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:34.670219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:34.712431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] MFA Rules not processed for user `8d404b2974f64aa8a6c32adfc3f7b0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:34.714582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:34.789962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 910/1803] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:34.804980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:34.806408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:34.828950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:34.833570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:34.854381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b43204-ef07-4a8c-a941-8820f1d3c69d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:34.867873 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:34.868788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:34.869370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:34.870331 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] PATH_INFO: `/v3/users/036d33e7289a409c9f9202eb2cbd9a59` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:34.871609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=036d33e7289a409c9f9202eb2cbd9a59)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:34.873535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:34.949684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036d33e7289a409c9f9202eb2cbd9a59'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:34.950325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036d33e7289a409c9f9202eb2cbd9a59'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:34.959156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036d33e7289a409c9f9202eb2cbd9a59'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:34.960015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036d33e7289a409c9f9202eb2cbd9a59'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:34.961253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44498-1dc3-46ff-9f12-afe7783455a5 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:34.968064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:34.968771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:34.969124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:34.969610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:34.971782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:34.974007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3d7f781a-684f-4afd-bb38-daa9532201cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036d33e7289a409c9f9202eb2cbd9a59'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:34.996178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:34.996814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 911/1804] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:34 2022] DELETE /identity/v3/users/036d33e7289a409c9f9202eb2cbd9a59 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:34.998354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:35.001194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:35.001599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:35.001981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:35.009176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 894/1805] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:35.026764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:35.028044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:35.100414 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:35.112031 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 912/1806] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:35.122339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:35.123943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:35.138067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccb92d-ec88-407e-be74-b21ed21227e0 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:35.234505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667819c2-3fbe-4b56-afb5-9c58e31567be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:35.235356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667819c2-3fbe-4b56-afb5-9c58e31567be service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:35.235785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667819c2-3fbe-4b56-afb5-9c58e31567be service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:35.236632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667819c2-3fbe-4b56-afb5-9c58e31567be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:35.248548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667819c2-3fbe-4b56-afb5-9c58e31567be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:35.251090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667819c2-3fbe-4b56-afb5-9c58e31567be service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:35.265567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 895/1807] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:35.643334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:35.645541 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:35.670609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97ebc8-323a-4083-82d3-f373ad6da578 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:35.682964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:35.683904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:35.685522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:35.686592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] PATH_INFO: `/v3/users/b705a02c504e46188c903bf2ce4952de` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:35.689411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b705a02c504e46188c903bf2ce4952de)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:35.694182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:35.774179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b705a02c504e46188c903bf2ce4952de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:35.774876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b705a02c504e46188c903bf2ce4952de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:35.785005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b705a02c504e46188c903bf2ce4952de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:35.785815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b705a02c504e46188c903bf2ce4952de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:35.802503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b705a02c504e46188c903bf2ce4952de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:35.832962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 913/1808] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:35 2022] DELETE /identity/v3/users/b705a02c504e46188c903bf2ce4952de => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:35.838851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:35.839439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:35.839905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:35.852800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:35.855037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:35.883457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:35.887580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:35.961892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b76e75-a9e9-42fb-be4d-d5db60f0e0e5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:35.967629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:35.968544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:35.968901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:35.969294 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] PATH_INFO: `/v3/projects/e8a98a4bedab4f87b69b14e54d75950d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:35.970672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a98a4bedab4f87b69b14e54d75950d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:35.972919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:36.011613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:36.035387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 896/1809] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:36.044254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:36.046204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:36.089834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a98a4bedab4f87b69b14e54d75950d', 'request_id': 'req-049f5e0b-dad7-4318-9f04-9b8c3865d23c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.114158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a98a4bedab4f87b69b14e54d75950d', 'request_id': 'req-049f5e0b-dad7-4318-9f04-9b8c3865d23c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.125557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a98a4bedab4f87b69b14e54d75950d', 'request_id': 'req-049f5e0b-dad7-4318-9f04-9b8c3865d23c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.166697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:36.174200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] The token cache is being invalidate because project e8a98a4bedab4f87b69b14e54d75950d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:36.175237 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a68fb-7f02-4c32-a6d9-4e6301e29772 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:36.175505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.176397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.178410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 914/1810] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:35 2022] DELETE /identity/v3/projects/e8a98a4bedab4f87b69b14e54d75950d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:36.184241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:36.188368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:36.249345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:36.250020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:36.250385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:36.250555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:36.256876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:36.308060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f5e0b-dad7-4318-9f04-9b8c3865d23c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:36.316089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:36.317226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:36.317601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:36.317972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] PATH_INFO: `/v3/users/37d4df7ea1494c2081bc2606d34c5863` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:36.319452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d4df7ea1494c2081bc2606d34c5863)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:36.322095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:36.367268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:36.368904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:36.390025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 897/1811] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:36.403715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d4df7ea1494c2081bc2606d34c5863'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.404404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d4df7ea1494c2081bc2606d34c5863'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.417603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d4df7ea1494c2081bc2606d34c5863'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.418480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d4df7ea1494c2081bc2606d34c5863'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.441846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce4a501c-11f1-4fab-bb55-4b19252053bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d4df7ea1494c2081bc2606d34c5863'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.459204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:36.460934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:36.477158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 915/1812] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:36 2022] DELETE /identity/v3/users/37d4df7ea1494c2081bc2606d34c5863 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:36.478378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3d68-40c5-4afb-a961-c49b4bae4333 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:36.553864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:36.555068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:36.555574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:36.556090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] PATH_INFO: `/v3/users/fd4f27a9b2b642bda08cb3ab05147e96` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:36.559606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4f27a9b2b642bda08cb3ab05147e96)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:36.562774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:36.570865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:36.634891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4f27a9b2b642bda08cb3ab05147e96'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.635804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4f27a9b2b642bda08cb3ab05147e96'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.645576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4f27a9b2b642bda08cb3ab05147e96'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.661051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4f27a9b2b642bda08cb3ab05147e96'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.662197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4f27a9b2b642bda08cb3ab05147e96'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:36.682490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 898/1813] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:36 2022] DELETE /identity/v3/users/fd4f27a9b2b642bda08cb3ab05147e96 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:36.686896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:36.687419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:36.687849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:36.708636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:36.709830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:36.770466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 916/1814] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:36.780559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:36.782164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:36.853034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7203-3c51-41fa-8d46-a499c570f28f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:36.857063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:36.858005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:36.858350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:36.859159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:36.870069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:36.871675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:36.881440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 899/1815] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:37.141061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:37.143196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:37.155775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447cb6a-59e1-4c8e-8b04-311b5a130b45 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:37.163285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:37.164070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:37.164409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:37.164752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] PATH_INFO: `/v3/projects/170a0b943fd847c3a46eb4c5e1df125c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:37.166798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=170a0b943fd847c3a46eb4c5e1df125c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:37.168390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:37.198538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:37.241792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170a0b943fd847c3a46eb4c5e1df125c', 'request_id': 'req-dd7674ec-e663-458a-8486-0d3d486c6957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.256919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170a0b943fd847c3a46eb4c5e1df125c', 'request_id': 'req-dd7674ec-e663-458a-8486-0d3d486c6957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.265738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170a0b943fd847c3a46eb4c5e1df125c', 'request_id': 'req-dd7674ec-e663-458a-8486-0d3d486c6957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.282778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] The token cache is being invalidate because project 170a0b943fd847c3a46eb4c5e1df125c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:37.283203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.284002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.289515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 917/1816] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:37 2022] DELETE /identity/v3/projects/170a0b943fd847c3a46eb4c5e1df125c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:37.290406 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:37.302414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d771d01-e6f4-406e-9a75-596bc2904a29 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:37.365534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:37.367436 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:37.414966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:37.415953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:37.416346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:37.416748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] PATH_INFO: `/v3/users/9d48c962ac1243ae9c645785febda074` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:37.418899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d48c962ac1243ae9c645785febda074)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:37.420955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:37.455437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:37.458721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7674ec-e663-458a-8486-0d3d486c6957 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:37.464354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:37.465042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:37.465254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:37.465531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] PATH_INFO: `/v3/projects/d271a44fc1314e67b6af3fe514d164d9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:37.466822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d271a44fc1314e67b6af3fe514d164d9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:37.469119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:37.487471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d48c962ac1243ae9c645785febda074'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.488823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d48c962ac1243ae9c645785febda074'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.498114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d48c962ac1243ae9c645785febda074'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.506460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d48c962ac1243ae9c645785febda074'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.507077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a6323ba7-1286-4901-909f-47dcbb945852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d48c962ac1243ae9c645785febda074'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.525429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 900/1817] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:37 2022] DELETE /identity/v3/users/9d48c962ac1243ae9c645785febda074 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:37.527180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713e1f-8314-404b-9b29-0a698ce8e333 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:37.527499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713e1f-8314-404b-9b29-0a698ce8e333 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:37.527775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713e1f-8314-404b-9b29-0a698ce8e333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:37.529767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd271a44fc1314e67b6af3fe514d164d9', 'request_id': 'req-ff712acd-028c-4df8-b336-62eb30101ec2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.536194 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd271a44fc1314e67b6af3fe514d164d9', 'request_id': 'req-ff712acd-028c-4df8-b336-62eb30101ec2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.544571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd271a44fc1314e67b6af3fe514d164d9', 'request_id': 'req-ff712acd-028c-4df8-b336-62eb30101ec2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.551754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-98713e1f-8314-404b-9b29-0a698ce8e333 None None] MFA Rules not processed for user `8d404b2974f64aa8a6c32adfc3f7b0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:37.553577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-98713e1f-8314-404b-9b29-0a698ce8e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:37.563389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] The token cache is being invalidate because project d271a44fc1314e67b6af3fe514d164d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:37.564249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.565120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.567448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 918/1818] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:37 2022] DELETE /identity/v3/projects/d271a44fc1314e67b6af3fe514d164d9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:37.568520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:37.569976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:37.614410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:37.660971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff712acd-028c-4df8-b336-62eb30101ec2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:37.708445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 901/1819] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:37.756810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:37.757733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:37.758145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:37.758474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] PATH_INFO: `/v3/projects/413eff6ee1534001b74994d9e5e3db06` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:37.759851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=413eff6ee1534001b74994d9e5e3db06)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:37.763650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:37.843486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413eff6ee1534001b74994d9e5e3db06', 'request_id': 'req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.852273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413eff6ee1534001b74994d9e5e3db06', 'request_id': 'req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.859799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413eff6ee1534001b74994d9e5e3db06', 'request_id': 'req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.874560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] The token cache is being invalidate because project 413eff6ee1534001b74994d9e5e3db06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:37.875036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.875772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:37.880571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 919/1820] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:37 2022] DELETE /identity/v3/projects/413eff6ee1534001b74994d9e5e3db06 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:37.882102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713e1f-8314-404b-9b29-0a698ce8e333 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:37.897264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713e1f-8314-404b-9b29-0a698ce8e333 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:37.973628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:37.974530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:37.974898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:37.975373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] PATH_INFO: `/v3/users/295502558219481b87f2fa74e9a53edc` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:37.977006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=295502558219481b87f2fa74e9a53edc)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:37.978956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:38.047884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '295502558219481b87f2fa74e9a53edc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.048786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '295502558219481b87f2fa74e9a53edc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.054220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:38.072005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '295502558219481b87f2fa74e9a53edc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.089704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '295502558219481b87f2fa74e9a53edc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.090521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '295502558219481b87f2fa74e9a53edc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.162138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 902/1821] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:37 2022] DELETE /identity/v3/users/295502558219481b87f2fa74e9a53edc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:38.272974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:38.274571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:38.290731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1a68e-2d95-4d3c-906d-b75bf12cbd06 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:38.377680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:38.378571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:38.378964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:38.379491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] PATH_INFO: `/v3/projects/a73521654f4241e1a33470db823c5809` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:38.380884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73521654f4241e1a33470db823c5809)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:38.383220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:38.397648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:38.447419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73521654f4241e1a33470db823c5809', 'request_id': 'req-8519164e-eae0-4688-82cf-b5b5a4a29338'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.455518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73521654f4241e1a33470db823c5809', 'request_id': 'req-8519164e-eae0-4688-82cf-b5b5a4a29338'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.464811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73521654f4241e1a33470db823c5809', 'request_id': 'req-8519164e-eae0-4688-82cf-b5b5a4a29338'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.489897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] The token cache is being invalidate because project a73521654f4241e1a33470db823c5809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:38.490327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.491353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.494021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 920/1822] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:38 2022] DELETE /identity/v3/projects/a73521654f4241e1a33470db823c5809 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:38.499148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:38.500747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:38.517497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682c64c-f4c4-4448-968f-2bfa3778b5b9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:38.611115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:38.611718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:38.611995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:38.612253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] PATH_INFO: `/v3/users/a658a884fc29403a8cbeb93b7e47586f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:38.613389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a658a884fc29403a8cbeb93b7e47586f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:38.615502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:38.674918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:38.683612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:38.707033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a658a884fc29403a8cbeb93b7e47586f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.707855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a658a884fc29403a8cbeb93b7e47586f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.711329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519164e-eae0-4688-82cf-b5b5a4a29338 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:38.718209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a658a884fc29403a8cbeb93b7e47586f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.728320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:38.729505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:38.729782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:38.730056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] PATH_INFO: `/v3/projects/100c0d9e03a84185aa72a50a8f4ff199` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:38.731520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100c0d9e03a84185aa72a50a8f4ff199)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:38.733132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a658a884fc29403a8cbeb93b7e47586f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.733739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:38.735811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de4444d2-dba2-46c3-ba2e-4ee144c2b840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a658a884fc29403a8cbeb93b7e47586f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.777583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 903/1823] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:38 2022] DELETE /identity/v3/users/a658a884fc29403a8cbeb93b7e47586f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:38.798135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100c0d9e03a84185aa72a50a8f4ff199', 'request_id': 'req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.806238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100c0d9e03a84185aa72a50a8f4ff199', 'request_id': 'req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.813772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100c0d9e03a84185aa72a50a8f4ff199', 'request_id': 'req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.835627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] The token cache is being invalidate because project 100c0d9e03a84185aa72a50a8f4ff199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:38.835934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.836929 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-620ac3f2-e339-45a5-a6f8-e8dc5cc41631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:38.843406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 921/1824] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:38 2022] DELETE /identity/v3/projects/100c0d9e03a84185aa72a50a8f4ff199 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:38.845251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b6bf-ec18-4ef1-b8f1-284619bc9c72 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:38.846168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b6bf-ec18-4ef1-b8f1-284619bc9c72 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:38.846443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b6bf-ec18-4ef1-b8f1-284619bc9c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:38.886162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7217b6bf-ec18-4ef1-b8f1-284619bc9c72 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:38.888604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7217b6bf-ec18-4ef1-b8f1-284619bc9c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:38.953782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:38.992544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 904/1825] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:39.001404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6db75f-beae-4923-8d5e-a193798c7139 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.001690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6db75f-beae-4923-8d5e-a193798c7139 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.001964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6db75f-beae-4923-8d5e-a193798c7139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.036642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cf6db75f-beae-4923-8d5e-a193798c7139 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:39.038446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cf6db75f-beae-4923-8d5e-a193798c7139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:39.049017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 922/1826] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:39.053591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b6bf-ec18-4ef1-b8f1-284619bc9c72 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.065754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b6bf-ec18-4ef1-b8f1-284619bc9c72 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.071642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.073176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.073693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.074158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.075814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:39.076336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.078281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.096065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 905/1827] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:39.096247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6db75f-beae-4923-8d5e-a193798c7139 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.106274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6db75f-beae-4923-8d5e-a193798c7139 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.111759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.113059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.113059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.113839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.114463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.116847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.177669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 923/1828] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/projects => generated 387 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:39.183883 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.198359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55510b4-7dd9-4508-b06d-fc2d12f390c4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.215153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.217352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:39.303859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:39.316247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44110cda-3a79-4a39-bbef-22e6d45e916f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.326514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.327162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.327465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.327736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] PATH_INFO: `/v3/projects/99949ef1619a4ca69f45022bde486919` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.328915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99949ef1619a4ca69f45022bde486919)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.330956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.344947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.345570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.345863 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.346110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.346737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.348724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.373580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 906/1829] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:39.377874 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.392862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27ef9-ad45-4105-a61a-6cf649fa5b0a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.398360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99949ef1619a4ca69f45022bde486919', 'request_id': 'req-5986ea82-4135-4ea9-9327-60be01580d52'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:39.406730 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99949ef1619a4ca69f45022bde486919', 'request_id': 'req-5986ea82-4135-4ea9-9327-60be01580d52'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:39.417116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99949ef1619a4ca69f45022bde486919', 'request_id': 'req-5986ea82-4135-4ea9-9327-60be01580d52'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:39.433570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] The token cache is being invalidate because project 99949ef1619a4ca69f45022bde486919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:39.433843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:39.434492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5986ea82-4135-4ea9-9327-60be01580d52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:39.436481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 924/1830] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:39 2022] DELETE /identity/v3/projects/99949ef1619a4ca69f45022bde486919 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:39.442654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ac57-c672-4059-b206-489353a65903 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.443056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ac57-c672-4059-b206-489353a65903 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.443495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ac57-c672-4059-b206-489353a65903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.468343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9e43ac57-c672-4059-b206-489353a65903 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:39.470281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9e43ac57-c672-4059-b206-489353a65903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:39.475889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.476543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.477353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.477353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.477878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.479744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.492793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 907/1831] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:39.497844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.500224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:39.517356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10a67-5f1f-48b9-9aee-fe0a354229f9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.522818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.523516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.523824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.524116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.524819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.526616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.557354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 908/1832] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:39.560571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.563134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 925/1833] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:39.573626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254c29-2cf1-4346-9ef9-0c8deec79bee None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.573933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254c29-2cf1-4346-9ef9-0c8deec79bee None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.574276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254c29-2cf1-4346-9ef9-0c8deec79bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.585066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90a6aa-bebc-4ffd-abfa-48f07c985560 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.591230 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299f3c4-d4a2-4832-939b-5a4ae540d92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.592070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299f3c4-d4a2-4832-939b-5a4ae540d92b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.592265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299f3c4-d4a2-4832-939b-5a4ae540d92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.592586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299f3c4-d4a2-4832-939b-5a4ae540d92b admin admin] PATH_INFO: `/v3/projects/f013df28437242928fd319b36b80324a/users/e6867151dc834e79bd9d2e91546c1ecd/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.599900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-aa254c29-2cf1-4346-9ef9-0c8deec79bee None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:39.603305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa254c29-2cf1-4346-9ef9-0c8deec79bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:39.618880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 926/1834] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:39.623722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa254c29-2cf1-4346-9ef9-0c8deec79bee None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.626337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299f3c4-d4a2-4832-939b-5a4ae540d92b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f013df28437242928fd319b36b80324a, user_id=e6867151dc834e79bd9d2e91546c1ecd, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.629721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299f3c4-d4a2-4832-939b-5a4ae540d92b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.637045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa254c29-2cf1-4346-9ef9-0c8deec79bee None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.643564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.644391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.644745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.645150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.645902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:39.646278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.648046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.659037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 909/1835] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:39 2022] PUT /identity/v3/projects/f013df28437242928fd319b36b80324a/users/e6867151dc834e79bd9d2e91546c1ecd/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:39.662387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 927/1836] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:39.666804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.672719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17652bc-a146-4ce8-918f-43180ab46ed2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.673056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17652bc-a146-4ce8-918f-43180ab46ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.673398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17652bc-a146-4ce8-918f-43180ab46ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.686116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48aac-e771-428c-a54a-2ae3ab3f4f6e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.730198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d17652bc-a146-4ce8-918f-43180ab46ed2 None None] MFA Rules not processed for user `e6867151dc834e79bd9d2e91546c1ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:39.731582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d17652bc-a146-4ce8-918f-43180ab46ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:39.768313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:39.814568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.815327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.815800 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.816431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.817279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.819853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.845819 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:39.860731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 928/1837] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:39.864897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.875246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 910/1838] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:39.877811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17652bc-a146-4ce8-918f-43180ab46ed2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:39.888938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17652bc-a146-4ce8-918f-43180ab46ed2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.980044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.980979 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.981361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.981867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.982620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:39.984650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210dd0e-1366-452d-a165-d99abe2de18e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:39.984786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:39.995823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:39.996543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:39.996952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:39.997289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:39.998017 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.000039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.020804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 911/1839] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:40.024737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.031393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 929/1840] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:39 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:40.035553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.047100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9217a6-644b-48e3-b4fd-7c23c76e0071 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.111735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f510-c948-4013-8450-aeaa66d5d5a6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.117528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.118236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.118610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.118983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.120028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.123005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.129834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.130641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.130763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.131086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.131758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.134055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.152919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 930/1841] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:40.158322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.171282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105b4b7-d50f-4c89-817e-54378cbdf98b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.176143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.176758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.177065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 912/1842] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:40 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:40.180053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.180344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.180997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.181118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.183252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.193195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec29754-10b9-44e6-a794-f7ead95a5a39 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.195920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 931/1843] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:40.200201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.204825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.205554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.205695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.206003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.206591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.208312 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.211148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9ff70-11dc-4ddf-8bcb-f103d1c2b5f1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.217592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910f506-0324-4e34-8bc9-016d9fc9ec29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.218589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f506-0324-4e34-8bc9-016d9fc9ec29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.218899 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f506-0324-4e34-8bc9-016d9fc9ec29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.220968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 913/1844] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:40.223371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f506-0324-4e34-8bc9-016d9fc9ec29 admin admin] PATH_INFO: `/v3/projects/7f212d9e7ab04122b557f9722d764099/users/6e394c32b5384761a2d0452ffb5b549c/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.225273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.238148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01835c49-0640-4e46-a476-ba0fe0b9147e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.245446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.248164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.248164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.248164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] PATH_INFO: `/v3/projects/778f954c15124d9ebf6e4ac261b8961b/users/dd789e9181154af7893174760530c784/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.256983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910f506-0324-4e34-8bc9-016d9fc9ec29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f212d9e7ab04122b557f9722d764099, user_id=6e394c32b5384761a2d0452ffb5b549c, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.259817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910f506-0324-4e34-8bc9-016d9fc9ec29 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.269593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778f954c15124d9ebf6e4ac261b8961b, user_id=dd789e9181154af7893174760530c784, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.273069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.276734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:40.278792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 932/1845] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:40 2022] PUT /identity/v3/projects/7f212d9e7ab04122b557f9722d764099/users/6e394c32b5384761a2d0452ffb5b549c/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:40.289720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5c954-9aed-473a-8080-7566ed046ba3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.289954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5c954-9aed-473a-8080-7566ed046ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.290173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5c954-9aed-473a-8080-7566ed046ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.298647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 914/1846] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:40 2022] PUT /identity/v3/projects/778f954c15124d9ebf6e4ac261b8961b/users/dd789e9181154af7893174760530c784/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:40.310929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.325393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0db5c954-9aed-473a-8080-7566ed046ba3 None None] MFA Rules not processed for user `6e394c32b5384761a2d0452ffb5b549c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:40.327363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0db5c954-9aed-473a-8080-7566ed046ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:40.362862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:40.387016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40562-8448-452d-8baa-a1ffecee4d77 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.391856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.392461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.392727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.393176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.393707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.395253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.405576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 933/1847] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:40.409413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 915/1848] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:40.410211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5c954-9aed-473a-8080-7566ed046ba3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.416766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.418413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:40.420644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5c954-9aed-473a-8080-7566ed046ba3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.425152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de75c1-e4a9-40ab-9a0e-3abb1ec5d7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.426078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de75c1-e4a9-40ab-9a0e-3abb1ec5d7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.426360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de75c1-e4a9-40ab-9a0e-3abb1ec5d7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.426637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de75c1-e4a9-40ab-9a0e-3abb1ec5d7d1 admin admin] PATH_INFO: `/v3/projects/778f954c15124d9ebf6e4ac261b8961b/users/dd789e9181154af7893174760530c784/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.428639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de75c1-e4a9-40ab-9a0e-3abb1ec5d7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778f954c15124d9ebf6e4ac261b8961b, user_id=dd789e9181154af7893174760530c784, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.431311 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de75c1-e4a9-40ab-9a0e-3abb1ec5d7d1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.442443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 934/1849] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:40 2022] PUT /identity/v3/projects/778f954c15124d9ebf6e4ac261b8961b/users/dd789e9181154af7893174760530c784/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:40.453641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.453919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.454185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.479616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab6aa1-e962-4d6b-b227-ae9c2656d8fe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.484315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] MFA Rules not processed for user `dd789e9181154af7893174760530c784`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:40.485520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:40.531739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638b150-ae75-42ad-8549-34d85e670ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.532365 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638b150-ae75-42ad-8549-34d85e670ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.532721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638b150-ae75-42ad-8549-34d85e670ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.533002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638b150-ae75-42ad-8549-34d85e670ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.554972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 935/1850] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:40.566642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:40.568020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:40.580339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea7dc0-cd95-4961-af27-476605502f8a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:40.583903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:40.584497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:40.584774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:40.585043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:40.594485 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.596004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.596415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638b150-ae75-42ad-8549-34d85e670ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:40.597850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638b150-ae75-42ad-8549-34d85e670ec2 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:40.605703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 936/1851] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:40.606609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 916/1852] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:41.260668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:41.340881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fd5b5-56d5-4240-a4ec-57f845b5fdd8 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:41.345501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:41.346120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:41.346392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:41.346657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] PATH_INFO: `/v3/users/e6867151dc834e79bd9d2e91546c1ecd` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:41.347774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6867151dc834e79bd9d2e91546c1ecd)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:41.349675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:41.383529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:41.398086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6867151dc834e79bd9d2e91546c1ecd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:41.398731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6867151dc834e79bd9d2e91546c1ecd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:41.406718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6867151dc834e79bd9d2e91546c1ecd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:41.408869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6867151dc834e79bd9d2e91546c1ecd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:41.421589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6867151dc834e79bd9d2e91546c1ecd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:41.436578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 937/1853] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:41 2022] DELETE /identity/v3/users/e6867151dc834e79bd9d2e91546c1ecd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:41.441106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:41.441594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:41.441705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:41.448797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:41.462843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:41.464113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:41.524377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 917/1854] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:41.533825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:41.535229 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:41.600002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425336b-35a5-4129-8d88-196eb41a8115 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:41.604092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:41.604649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:41.604918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:41.605192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:41.614368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:41.615834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:41.632970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 938/1855] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:41.925051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:41.927922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:41.960236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb417849-3d6e-4fe4-96f9-321f8dab545c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:41.966118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:41.967178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:41.967592 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:41.968042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] PATH_INFO: `/v3/projects/f013df28437242928fd319b36b80324a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:41.969927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f013df28437242928fd319b36b80324a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:41.972029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:42.106685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f013df28437242928fd319b36b80324a', 'request_id': 'req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.119277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f013df28437242928fd319b36b80324a', 'request_id': 'req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.135606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f013df28437242928fd319b36b80324a', 'request_id': 'req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.170380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] The token cache is being invalidate because project f013df28437242928fd319b36b80324a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:42.170683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.171501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.173421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 918/1856] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:41 2022] DELETE /identity/v3/projects/f013df28437242928fd319b36b80324a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:42.177241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:42.193164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd16b-e600-4398-a836-fd213f9036a7 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:42.301611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:42.302217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:42.302501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:42.303055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] PATH_INFO: `/v3/users/dd789e9181154af7893174760530c784` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:42.304367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd789e9181154af7893174760530c784)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:42.306455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:42.356032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd789e9181154af7893174760530c784'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.356693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd789e9181154af7893174760530c784'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.361898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:42.363963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd789e9181154af7893174760530c784'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.364688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd789e9181154af7893174760530c784'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.378797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce54bab7-3788-44de-a9a9-a4e39612baca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd789e9181154af7893174760530c784'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.399297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 939/1857] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:42 2022] DELETE /identity/v3/users/dd789e9181154af7893174760530c784 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:42.839737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:42.841251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:42.856268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d23bd-971a-4108-88ae-7b5f8e19db8f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:42.932704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:42.933336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:42.933627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:42.933878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] PATH_INFO: `/v3/projects/778f954c15124d9ebf6e4ac261b8961b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:42.934972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] RBAC: Authorizing `identity:delete_project(project_id=778f954c15124d9ebf6e4ac261b8961b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:42.936958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:42.969860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:42.986724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778f954c15124d9ebf6e4ac261b8961b', 'request_id': 'req-a50da732-3205-4238-b0c0-6ae1caee649c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.992497 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778f954c15124d9ebf6e4ac261b8961b', 'request_id': 'req-a50da732-3205-4238-b0c0-6ae1caee649c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:42.999897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778f954c15124d9ebf6e4ac261b8961b', 'request_id': 'req-a50da732-3205-4238-b0c0-6ae1caee649c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.011654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] The token cache is being invalidate because project 778f954c15124d9ebf6e4ac261b8961b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:43.011941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.012588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a50da732-3205-4238-b0c0-6ae1caee649c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.014079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 919/1858] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:42 2022] DELETE /identity/v3/projects/778f954c15124d9ebf6e4ac261b8961b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:43.019447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77ddd-646c-4d2f-8f3a-7a2c2025c08d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.019732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77ddd-646c-4d2f-8f3a-7a2c2025c08d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.020006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77ddd-646c-4d2f-8f3a-7a2c2025c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.040041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b6a77ddd-646c-4d2f-8f3a-7a2c2025c08d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:43.041326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b6a77ddd-646c-4d2f-8f3a-7a2c2025c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.113149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 940/1859] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:43.120675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e55258-990b-489d-9c52-0b367aec147b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.127556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e55258-990b-489d-9c52-0b367aec147b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.127848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e55258-990b-489d-9c52-0b367aec147b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.177095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e0e55258-990b-489d-9c52-0b367aec147b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:43.178559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e0e55258-990b-489d-9c52-0b367aec147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.192067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 920/1860] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:43.208425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a77ddd-646c-4d2f-8f3a-7a2c2025c08d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.218737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a77ddd-646c-4d2f-8f3a-7a2c2025c08d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.223482 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.224101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.224379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.224647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.225289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:43.225586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.227180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.243522 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 941/1861] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:43.245074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e55258-990b-489d-9c52-0b367aec147b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.256693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e55258-990b-489d-9c52-0b367aec147b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.262281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.262893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.263168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.263546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.264188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.266262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.298287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 921/1862] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:43 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:43.302490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.315087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5803e-9086-4733-a944-253f4b3c3757 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.396039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.396787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.397066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.397333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.397970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.400001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.425806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.427846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.434298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 942/1863] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:43 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:43.438241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.450201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40af78a-d8db-453d-ac10-367d35b104ec admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.451527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfce342-3201-4174-b6d1-454466cdfabe admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.455284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.455920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.456192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.456460 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] PATH_INFO: `/v3/users/6e394c32b5384761a2d0452ffb5b549c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.456887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.457600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e394c32b5384761a2d0452ffb5b549c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.457719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.458446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.459945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.459945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.460600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.463673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.482409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 943/1864] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:43.483525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.497424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f7914-87a3-453a-adb1-0a58239565f5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.505370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.506027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.506304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.506983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.507332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.509508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.525957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 944/1865] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:43.530444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.544116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713014-4921-4abd-b756-dfb5626b2dfb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.556182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5848f-4b0e-4b05-927b-b2271988ea44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.558152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5848f-4b0e-4b05-927b-b2271988ea44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.558599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5848f-4b0e-4b05-927b-b2271988ea44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.559271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5848f-4b0e-4b05-927b-b2271988ea44 admin admin] PATH_INFO: `/v3/projects/b7e54a5722064351bf4b3c51ba01b5e5/users/8326554286c54631b31659e0cd3add52/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.560288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e394c32b5384761a2d0452ffb5b549c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.562747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e394c32b5384761a2d0452ffb5b549c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.569842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e394c32b5384761a2d0452ffb5b549c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.581969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e394c32b5384761a2d0452ffb5b549c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.583013 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24abc72d-f2c7-4bce-8a54-ca35be760f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e394c32b5384761a2d0452ffb5b549c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:43.606989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5848f-4b0e-4b05-927b-b2271988ea44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7e54a5722064351bf4b3c51ba01b5e5, user_id=8326554286c54631b31659e0cd3add52, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.613853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5848f-4b0e-4b05-927b-b2271988ea44 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.621375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 922/1866] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:43 2022] DELETE /identity/v3/users/6e394c32b5384761a2d0452ffb5b549c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:43.626275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.626658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.626912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.642381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 945/1867] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:43 2022] PUT /identity/v3/projects/b7e54a5722064351bf4b3c51ba01b5e5/users/8326554286c54631b31659e0cd3add52/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:43.653624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.653947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.654179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.662618 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:43.671359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:43.674075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.699755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] MFA Rules not processed for user `8326554286c54631b31659e0cd3add52`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:43.703576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.773911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:43.789691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 923/1868] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:43.800093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 946/1869] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:43.801578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.803352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.812122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.813777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.883407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820b59-3b2a-419d-b702-416f0c5ce067 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.888368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa01ed0-c627-4018-8073-061e5e4fcfcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.889668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01ed0-c627-4018-8073-061e5e4fcfcb service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.890418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01ed0-c627-4018-8073-061e5e4fcfcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.904606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01ed0-c627-4018-8073-061e5e4fcfcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.907152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc04c-4b6f-48be-ab8d-a6208d7d3d6b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.914466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.914986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.915311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.915717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.926434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa01ed0-c627-4018-8073-061e5e4fcfcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.927437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.927947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa01ed0-c627-4018-8073-061e5e4fcfcb service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.929475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:43.941159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 924/1870] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:43.944432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 947/1871] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:43.965580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:43.967757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:43.983254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1fa21-8082-4482-86e7-b5d171132794 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:43.993292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:43.994190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:43.994561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:43.994938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] PATH_INFO: `/v3/users/8326554286c54631b31659e0cd3add52` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:43.996199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8326554286c54631b31659e0cd3add52)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:43.998583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:44.054138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8326554286c54631b31659e0cd3add52'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.054964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8326554286c54631b31659e0cd3add52'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.063286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8326554286c54631b31659e0cd3add52'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.072339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8326554286c54631b31659e0cd3add52'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.072660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8326554286c54631b31659e0cd3add52'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.099265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 925/1872] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:43 2022] DELETE /identity/v3/users/8326554286c54631b31659e0cd3add52 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:44.101365 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.101746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.102084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.123749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:44.124908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:44.223553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 948/1873] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:44.225725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:44.227580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:44.327421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991127b2-57ad-4767-87c7-e8bc05f8d3d0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:44.334502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9d5fe-ac1c-4390-ac96-02f444ac42cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:44.335324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9d5fe-ac1c-4390-ac96-02f444ac42cc service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.335703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9d5fe-ac1c-4390-ac96-02f444ac42cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.336089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9d5fe-ac1c-4390-ac96-02f444ac42cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.352195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9d5fe-ac1c-4390-ac96-02f444ac42cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:44.354689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9d5fe-ac1c-4390-ac96-02f444ac42cc service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:44.366895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 926/1874] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:44.517389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:44.519048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:44.534164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63f686-34e0-4610-9030-92cb1366c220 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:44.543413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:44.545051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.545548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.546050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] PATH_INFO: `/v3/projects/7f212d9e7ab04122b557f9722d764099` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.547878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f212d9e7ab04122b557f9722d764099)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:44.551265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:44.595075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:44.611666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f212d9e7ab04122b557f9722d764099', 'request_id': 'req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.618645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f212d9e7ab04122b557f9722d764099', 'request_id': 'req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.626851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f212d9e7ab04122b557f9722d764099', 'request_id': 'req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.641534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] The token cache is being invalidate because project 7f212d9e7ab04122b557f9722d764099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:44.641855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.642518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-21551fb4-1089-47c6-9bf2-69d7d23e6c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.647642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 949/1875] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:44 2022] DELETE /identity/v3/projects/7f212d9e7ab04122b557f9722d764099 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:44.649571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7e4e4-fd7e-4f9d-a5c4-494e519938e9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.649862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7e4e4-fd7e-4f9d-a5c4-494e519938e9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.650135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7e4e4-fd7e-4f9d-a5c4-494e519938e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.677301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e0e7e4e4-fd7e-4f9d-a5c4-494e519938e9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:44.679425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e0e7e4e4-fd7e-4f9d-a5c4-494e519938e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:44.693138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:44.753541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 927/1876] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:44.756302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.756302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.756302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.776919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:44.778226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:44.791263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 950/1877] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:44.793846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7e4e4-fd7e-4f9d-a5c4-494e519938e9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:44.797985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:44.800402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:44.804443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7e4e4-fd7e-4f9d-a5c4-494e519938e9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:44.811740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:44.812356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.812632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.812902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.813541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:44.813843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:44.815637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:44.821002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904df13-404e-4bba-9ef2-303a1de4526d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:44.826687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:44.827352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.827629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.827897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] PATH_INFO: `/v3/projects/b7e54a5722064351bf4b3c51ba01b5e5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.829023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7e54a5722064351bf4b3c51ba01b5e5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:44.829661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 928/1878] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:04:44.830937 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:44.833873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:44.844249 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f159c26-b55f-42ba-b532-f917af35b034 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:44.849476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:44.850085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.850361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.850632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.852924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:44.859757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:44.894624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7e54a5722064351bf4b3c51ba01b5e5', 'request_id': 'req-0526ac3e-f62b-4263-9f01-04d720ee891b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.902710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7e54a5722064351bf4b3c51ba01b5e5', 'request_id': 'req-0526ac3e-f62b-4263-9f01-04d720ee891b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.914273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 929/1879] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:44 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:44.916782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7e54a5722064351bf4b3c51ba01b5e5', 'request_id': 'req-0526ac3e-f62b-4263-9f01-04d720ee891b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.917532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:44.937354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] The token cache is being invalidate because project b7e54a5722064351bf4b3c51ba01b5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:44.937946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.938821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0526ac3e-f62b-4263-9f01-04d720ee891b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:44.940806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 951/1880] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:44 2022] DELETE /identity/v3/projects/b7e54a5722064351bf4b3c51ba01b5e5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:44.945794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d28d8-ffe1-460b-94ee-ed463fb0c531 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:44.946088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d28d8-ffe1-460b-94ee-ed463fb0c531 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:44.946368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d28d8-ffe1-460b-94ee-ed463fb0c531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:44.973148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f79d28d8-ffe1-460b-94ee-ed463fb0c531 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:44.974495 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f79d28d8-ffe1-460b-94ee-ed463fb0c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:44.999857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4af1d-bcc9-4fce-aa70-9b5f953bbeb1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.005520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.006176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.006580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.006912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.007657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.009725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.040211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 930/1881] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:44 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:45.041338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.042985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:45.056137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fcdf1-9208-470f-bff4-04d83948bd30 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.061823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.062279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.062487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.062873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.063577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.065324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.081149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 931/1882] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:45.082161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.083519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 952/1883] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:45.092167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2262447-11ca-4a4b-811e-80b53686b13d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.092573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2262447-11ca-4a4b-811e-80b53686b13d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.092970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2262447-11ca-4a4b-811e-80b53686b13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.102437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86af1b-cf43-4940-890a-a3902cbc6757 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.108671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.109213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.109491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.109763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.110405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.112039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.116895 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a2262447-11ca-4a4b-811e-80b53686b13d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:45.118100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a2262447-11ca-4a4b-811e-80b53686b13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:45.125960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 932/1884] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:45.130024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.130482 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 953/1885] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:45.134508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2262447-11ca-4a4b-811e-80b53686b13d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.139928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cb6b0-5443-4368-a31f-f4bab80e78d7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.143408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2262447-11ca-4a4b-811e-80b53686b13d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.144684 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10910ebd-53a2-4678-9cc9-c6c42eadf077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.145391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10910ebd-53a2-4678-9cc9-c6c42eadf077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.145732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10910ebd-53a2-4678-9cc9-c6c42eadf077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.146068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10910ebd-53a2-4678-9cc9-c6c42eadf077 admin admin] PATH_INFO: `/v3/projects/db8098f6f78c49fa843d34d69b002daa/users/c09a31533d7847fea61682ff032f838b/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.148469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.149125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.149488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.149831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.151266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:45.151266 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.152646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.164004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10910ebd-53a2-4678-9cc9-c6c42eadf077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8098f6f78c49fa843d34d69b002daa, user_id=c09a31533d7847fea61682ff032f838b, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.164942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 954/1886] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:04:45.167359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10910ebd-53a2-4678-9cc9-c6c42eadf077 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.169664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.180810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471ad1b-c9c3-47ae-a97a-8cbbadfe97d9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.184491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 933/1887] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:45 2022] PUT /identity/v3/projects/db8098f6f78c49fa843d34d69b002daa/users/c09a31533d7847fea61682ff032f838b/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:45.189352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.189645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.189898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.192601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:45.209055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:45.210242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:45.230269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:45.240745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.241355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.241638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.241906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.242526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.245018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.275823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 955/1888] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:45 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:45.276037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 934/1889] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:45.276774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.282916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.284329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:45.338237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8be422-5e66-4d72-8c1d-c780c77007c8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.343348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.343959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.344062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e551fe-f227-4feb-94ba-5f4df5385538 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.344236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.344640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.345280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.347187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.373925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 956/1890] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:45 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:45.378134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.398185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.398790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.399065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.399378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.401500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd80e-c579-4a34-ad06-86af8b967052 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.406317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.406960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.407604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.407604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.408148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.409728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.410379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.412379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.421014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 957/1891] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:45.424706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.426246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 935/1892] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:04:45.434872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0ab3a-25a0-4f3c-b8cd-b8c665a9b6b9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.440413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87d906-10a8-4010-8d35-a516144d3e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.441080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87d906-10a8-4010-8d35-a516144d3e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.441424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87d906-10a8-4010-8d35-a516144d3e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.441762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87d906-10a8-4010-8d35-a516144d3e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.442572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87d906-10a8-4010-8d35-a516144d3e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.444297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87d906-10a8-4010-8d35-a516144d3e19 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.457765 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 958/1893] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:45.462183 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.473848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc3dfe-c8b2-42a5-ae21-ef2189b5f592 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.478839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.479616 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.479960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.480639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] PATH_INFO: `/v3/projects/58fdf05cfb044a2aa98bc12a38fb76cd/users/2e07169459e44b079cbcc5157874c89a/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.499334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fdf05cfb044a2aa98bc12a38fb76cd, user_id=2e07169459e44b079cbcc5157874c89a, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.503438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.526491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 936/1894] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:45 2022] PUT /identity/v3/projects/58fdf05cfb044a2aa98bc12a38fb76cd/users/2e07169459e44b079cbcc5157874c89a/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:45.531826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6efaa-3fad-477e-b88a-d23e8f89c1f5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.532299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6efaa-3fad-477e-b88a-d23e8f89c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.532916 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6efaa-3fad-477e-b88a-d23e8f89c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.560301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-54b6efaa-3fad-477e-b88a-d23e8f89c1f5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:45.562578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-54b6efaa-3fad-477e-b88a-d23e8f89c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:45.661193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 959/1895] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:45.672511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:45.674634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:45.736150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e7672-6861-40cf-86da-ea8d0f02aec0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:45.739957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:45.740397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:45.740672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:45.740942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:45.751043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:45.752774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:45.763534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 937/1896] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:46.657551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36854b09-b495-4a77-847f-c548b9433330 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:46.658083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36854b09-b495-4a77-847f-c548b9433330 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:46.658083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36854b09-b495-4a77-847f-c548b9433330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:46.683814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:46.695777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-36854b09-b495-4a77-847f-c548b9433330 None None] MFA Rules not processed for user `2e07169459e44b079cbcc5157874c89a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:46.697910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-36854b09-b495-4a77-847f-c548b9433330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:46.736390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:46.737843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:46.749306 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:46.761856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b4950-05e9-4023-97c1-b7480b235bd5 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:46.769777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:46.770696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:46.771076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:46.771526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] PATH_INFO: `/v3/users/50d81a392850498f9cb6bf2bdd6e97bd` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:46.772833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d81a392850498f9cb6bf2bdd6e97bd)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:46.776575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:46.812117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 960/1897] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:46.816655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36854b09-b495-4a77-847f-c548b9433330 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:46.818981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-36854b09-b495-4a77-847f-c548b9433330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:46.848197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d81a392850498f9cb6bf2bdd6e97bd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:46.848874 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d81a392850498f9cb6bf2bdd6e97bd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:46.860222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d81a392850498f9cb6bf2bdd6e97bd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:46.873898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d81a392850498f9cb6bf2bdd6e97bd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:46.874730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-383cba68-dac0-406e-908b-a4446ec54e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d81a392850498f9cb6bf2bdd6e97bd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:46.903791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 938/1898] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:04:46 2022] DELETE /identity/v3/users/50d81a392850498f9cb6bf2bdd6e97bd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:46.908362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:46.908795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:46.909074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:46.921805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36854b09-b495-4a77-847f-c548b9433330 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:46.941375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:46.943094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:47.003238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:47.004353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:47.004635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:47.004908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:47.008400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:47.049146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 939/1899] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:04:47.069766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:47.071312 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:47.159322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:47.161653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:47.198829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 961/1900] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:47.207916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ffaa8-5763-4b83-97ff-5697c7a620ee None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:47.219292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53b633-e5dd-4f76-b4ab-d7291a14db0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:47.219911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53b633-e5dd-4f76-b4ab-d7291a14db0e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:47.220190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53b633-e5dd-4f76-b4ab-d7291a14db0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:47.220454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53b633-e5dd-4f76-b4ab-d7291a14db0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:47.244721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53b633-e5dd-4f76-b4ab-d7291a14db0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:47.246481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53b633-e5dd-4f76-b4ab-d7291a14db0e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:47.259189 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 940/1901] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:47.970516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:47.972024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:47.986941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902795-451a-4ea1-b6bf-df94d6c46357 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:47.992427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:47.993265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:47.993568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:47.993841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] PATH_INFO: `/v3/projects/1fd3f16846a2498ba61ac29a2df44957` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:47.996094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd3f16846a2498ba61ac29a2df44957)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:47.998004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.001007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:48.060625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd3f16846a2498ba61ac29a2df44957', 'request_id': 'req-2bf81039-5afe-48cd-bd47-a9ec89522cb2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:48.069498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd3f16846a2498ba61ac29a2df44957', 'request_id': 'req-2bf81039-5afe-48cd-bd47-a9ec89522cb2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:48.078575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd3f16846a2498ba61ac29a2df44957', 'request_id': 'req-2bf81039-5afe-48cd-bd47-a9ec89522cb2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:48.093467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] The token cache is being invalidate because project 1fd3f16846a2498ba61ac29a2df44957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:04:48.093790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:48.103077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2bf81039-5afe-48cd-bd47-a9ec89522cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:04:48.106675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 962/1902] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:04:47 2022] DELETE /identity/v3/projects/1fd3f16846a2498ba61ac29a2df44957 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:48.111935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4612ee6-83f7-4ea3-b4e1-9b83edf77fed None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.112598 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4612ee6-83f7-4ea3-b4e1-9b83edf77fed None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.112832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4612ee6-83f7-4ea3-b4e1-9b83edf77fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.142281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f4612ee6-83f7-4ea3-b4e1-9b83edf77fed None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:48.143563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f4612ee6-83f7-4ea3-b4e1-9b83edf77fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:48.169400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:48.235633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 941/1903] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:48.242077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad803ad2-fe39-4feb-9e75-07c68562b172 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.242381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad803ad2-fe39-4feb-9e75-07c68562b172 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.242647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad803ad2-fe39-4feb-9e75-07c68562b172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.268319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad803ad2-fe39-4feb-9e75-07c68562b172 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:48.269985 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad803ad2-fe39-4feb-9e75-07c68562b172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:48.281256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 963/1904] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:48.285472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4612ee6-83f7-4ea3-b4e1-9b83edf77fed None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:48.302885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4612ee6-83f7-4ea3-b4e1-9b83edf77fed None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:48.337570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:48.338445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.338724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.339054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.339742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:04:48.340042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:48.341972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.354606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 942/1905] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:04:48.360359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad803ad2-fe39-4feb-9e75-07c68562b172 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:48.373306 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad803ad2-fe39-4feb-9e75-07c68562b172 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:48.385464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:48.386865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.387243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.387609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.388343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:48.390483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.430245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 964/1906] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:48 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:48.431305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:48.442106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ed9b1-f636-4602-89a8-b3cf468ddb29 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:48.507494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:48.508331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.508786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.509058 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.509731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:48.512324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.539915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 943/1907] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:48 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:04:48.541303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:48.551591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ae126-6450-477c-93f1-0a421852855b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:48.556413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:48.557742 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.558036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.558375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.559074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:48.560923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.572122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 965/1908] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:48.576797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:48.586891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99980cb-3f7a-4118-847b-0a939e789f63 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:48.592897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c2ad2-7b28-4698-9ce4-467adb78e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:48.593582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2ad2-7b28-4698-9ce4-467adb78e16b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.594016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2ad2-7b28-4698-9ce4-467adb78e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.594286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2ad2-7b28-4698-9ce4-467adb78e16b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.595149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c2ad2-7b28-4698-9ce4-467adb78e16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:48.596904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c2ad2-7b28-4698-9ce4-467adb78e16b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.611397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 944/1909] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:48.612802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:48.623467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d347e0f-1b46-47ca-b3cb-0fb668e27bee admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:48.628317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe61e5f-329d-4b33-8add-8879bb067159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:48.628991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe61e5f-329d-4b33-8add-8879bb067159 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.629270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe61e5f-329d-4b33-8add-8879bb067159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.629542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe61e5f-329d-4b33-8add-8879bb067159 admin admin] PATH_INFO: `/v3/projects/91958c61267b494aabe3e63a13d6f452/users/7a2a6b0ec599444385bde13a429a7ecb/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.651023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe61e5f-329d-4b33-8add-8879bb067159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91958c61267b494aabe3e63a13d6f452, user_id=7a2a6b0ec599444385bde13a429a7ecb, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:48.653889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe61e5f-329d-4b33-8add-8879bb067159 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.668427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 966/1910] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:48 2022] PUT /identity/v3/projects/91958c61267b494aabe3e63a13d6f452/users/7a2a6b0ec599444385bde13a429a7ecb/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:04:48.680321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f138-9971-4415-a540-5421f58e16ab None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.681289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f138-9971-4415-a540-5421f58e16ab None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.683256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f138-9971-4415-a540-5421f58e16ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.704855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.705139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.705412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.726816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] MFA Rules not processed for user `2e07169459e44b079cbcc5157874c89a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:48.727404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cbb4f138-9971-4415-a540-5421f58e16ab None None] MFA Rules not processed for user `7a2a6b0ec599444385bde13a429a7ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:48.728066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:48.728743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbb4f138-9971-4415-a540-5421f58e16ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:48.805727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 967/1911] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:48.823238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 945/1912] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:48.834633 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:48.836099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:48.876168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:48.907131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7e08c-cb6b-40a7-9737-fb0c11c30e64 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:48.912049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b3b92-d4da-41c2-8ce6-265293c02001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:48.913008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b3b92-d4da-41c2-8ce6-265293c02001 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:48.913397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b3b92-d4da-41c2-8ce6-265293c02001 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:48.913708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b3b92-d4da-41c2-8ce6-265293c02001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:48.923827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b3b92-d4da-41c2-8ce6-265293c02001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:48.925268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b3b92-d4da-41c2-8ce6-265293c02001 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:48.943359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 968/1913] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:04:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:04:50.397464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:50.397986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:50.397986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:50.423401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] MFA Rules not processed for user `7a2a6b0ec599444385bde13a429a7ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:50.425246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:50.441552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 946/1914] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:52.378174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195646-7f30-420d-93bd-c5903e17a25c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:52.378977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195646-7f30-420d-93bd-c5903e17a25c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:52.379434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195646-7f30-420d-93bd-c5903e17a25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:52.448178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-33195646-7f30-420d-93bd-c5903e17a25c None None] MFA Rules not processed for user `c09a31533d7847fea61682ff032f838b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:04:52.452419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-33195646-7f30-420d-93bd-c5903e17a25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:52.582435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 969/1915] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:04:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:04:52.586686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:52.588270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:04:52.601579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:04:52.692796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2333e-174e-4d95-9729-ba678b6e30bc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:52.698920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:52.700238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:52.700527 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:52.700892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:52.701458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:52.703417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:52.736498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 947/1916] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:04:52 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:52.740330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195646-7f30-420d-93bd-c5903e17a25c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:52.756723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195646-7f30-420d-93bd-c5903e17a25c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:52.819756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:52.820370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:52.820846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:52.820955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:52.821544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:52.824442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:52.847924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 970/1917] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:04:52 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:04:52.851690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:52.862491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dec9f-e9df-4ebc-ad73-ea1915f48238 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:52.867192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:52.867822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:52.868099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:52.868389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:52.869109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:52.871261 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:52.885960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 948/1918] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:04:52.887561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:52.898528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf049e9-4ae3-4a87-8b3d-573e31aae90d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:52.906083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:52.906771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:52.907624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:52.907624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] PATH_INFO: `/v3/projects/a5c5bdd7cef34826843be339ab43e251/users/62038532ab4e4398ab43e593c7ec1224/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:52.926140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c5bdd7cef34826843be339ab43e251, user_id=62038532ab4e4398ab43e593c7ec1224, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:52.928969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:52.946405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 971/1919] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:52 2022] PUT /identity/v3/projects/a5c5bdd7cef34826843be339ab43e251/users/62038532ab4e4398ab43e593c7ec1224/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:04:52.951304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:52.961987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbba17e-2213-4136-93e9-05a3be1ccdd4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:53.068713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28b897-5360-496f-94b1-f40214c13d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:53.069339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b897-5360-496f-94b1-f40214c13d55 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:53.069927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b897-5360-496f-94b1-f40214c13d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:53.070392 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b897-5360-496f-94b1-f40214c13d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:53.071142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28b897-5360-496f-94b1-f40214c13d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:53.073453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28b897-5360-496f-94b1-f40214c13d55 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:53.084843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 949/1920] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:04:53.089615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:04:53.103019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ebc56-2b64-4800-8519-602d55454fb0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:04:53.109838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:04:53.110698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:04:53.111119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:04:53.111428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] PATH_INFO: `/v3/projects/a5c5bdd7cef34826843be339ab43e251/users/62038532ab4e4398ab43e593c7ec1224/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:04:53.113684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c5bdd7cef34826843be339ab43e251, user_id=62038532ab4e4398ab43e593c7ec1224, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:04:53.121989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:04:53.136146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 972/1921] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:04:53 2022] PUT /identity/v3/projects/a5c5bdd7cef34826843be339ab43e251/users/62038532ab4e4398ab43e593c7ec1224/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:05:01.377025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82124b-3c1a-40f2-bd48-c43abf3d7899 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:01.377025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82124b-3c1a-40f2-bd48-c43abf3d7899 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:01.377601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82124b-3c1a-40f2-bd48-c43abf3d7899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:01.408992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7e82124b-3c1a-40f2-bd48-c43abf3d7899 None None] MFA Rules not processed for user `62038532ab4e4398ab43e593c7ec1224`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:01.410484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7e82124b-3c1a-40f2-bd48-c43abf3d7899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:01.488181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 950/1922] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:01.503770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:01.505688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:01.529320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:01.575248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bd1f0-6bd3-4f96-bb9e-a0384265e221 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:01.581476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:01.582592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:01.582898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:01.583172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:01.656132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:01.657774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:01.671148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 973/1923] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:05:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:05:01.842666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed32203-1b00-45a7-a137-e134f3152249 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:01.842953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed32203-1b00-45a7-a137-e134f3152249 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:01.843309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed32203-1b00-45a7-a137-e134f3152249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:01.854068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:01.867065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2ed32203-1b00-45a7-a137-e134f3152249 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:01.870692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2ed32203-1b00-45a7-a137-e134f3152249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:01.934106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 951/1924] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:05:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:02.016553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:02.022048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:02.103901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb3469-dc48-4981-86d0-2202dce96bb3 service glance] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:02.110407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544deb35-f844-4b07-9862-fc45936e7338 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:02.112033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544deb35-f844-4b07-9862-fc45936e7338 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:02.113839 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544deb35-f844-4b07-9862-fc45936e7338 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:02.114321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544deb35-f844-4b07-9862-fc45936e7338 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:02.128160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544deb35-f844-4b07-9862-fc45936e7338 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:02.130417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544deb35-f844-4b07-9862-fc45936e7338 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:02.140713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 974/1925] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:05:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:05:11.926470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=23 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:12.137052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=72 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:13.880475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913552bc-a065-4aae-bcf5-20c033f013ba None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:13.881011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913552bc-a065-4aae-bcf5-20c033f013ba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:13.881011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913552bc-a065-4aae-bcf5-20c033f013ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:13.900664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-913552bc-a065-4aae-bcf5-20c033f013ba None None] MFA Rules not processed for user `c09a31533d7847fea61682ff032f838b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:13.901860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-913552bc-a065-4aae-bcf5-20c033f013ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:13.908672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 952/1926] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:19.195085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a2fbe-bf5c-4955-9811-96425d7f7a60 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:19.195625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a2fbe-bf5c-4955-9811-96425d7f7a60 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:19.195625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a2fbe-bf5c-4955-9811-96425d7f7a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:19.214999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d55a2fbe-bf5c-4955-9811-96425d7f7a60 None None] MFA Rules not processed for user `2e07169459e44b079cbcc5157874c89a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:19.216321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d55a2fbe-bf5c-4955-9811-96425d7f7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:19.271270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 975/1927] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:21.027785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2bb46-4797-4065-aeef-442a4c2e7159 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:21.029448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2bb46-4797-4065-aeef-442a4c2e7159 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:21.029937 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2bb46-4797-4065-aeef-442a4c2e7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:21.062521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cce2bb46-4797-4065-aeef-442a4c2e7159 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:21.064008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cce2bb46-4797-4065-aeef-442a4c2e7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:21.080736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 953/1928] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:05:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:21.094682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a2fbe-bf5c-4955-9811-96425d7f7a60 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:21.113928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a2fbe-bf5c-4955-9811-96425d7f7a60 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:21.126898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133175d-5bab-41fb-a8f7-10bda0524cc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:21.128018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133175d-5bab-41fb-a8f7-10bda0524cc2 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:21.128749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133175d-5bab-41fb-a8f7-10bda0524cc2 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:21.129347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133175d-5bab-41fb-a8f7-10bda0524cc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:21.141816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133175d-5bab-41fb-a8f7-10bda0524cc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:21.144547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133175d-5bab-41fb-a8f7-10bda0524cc2 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:21.161178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 976/1929] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:05:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:05:21.565855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618eea-d24a-4239-8c50-498ca42aaf86 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:21.569717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618eea-d24a-4239-8c50-498ca42aaf86 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:21.570324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618eea-d24a-4239-8c50-498ca42aaf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:21.613402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3a618eea-d24a-4239-8c50-498ca42aaf86 None None] MFA Rules not processed for user `c987a5a52fc641f78054ac0218edcd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:21.615204 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3a618eea-d24a-4239-8c50-498ca42aaf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:21.742533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 954/1930] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:23.564719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8942da2-dfc0-4416-8f1d-16ff45ab4df0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:23.565814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8942da2-dfc0-4416-8f1d-16ff45ab4df0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:23.566252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8942da2-dfc0-4416-8f1d-16ff45ab4df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:23.603554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b8942da2-dfc0-4416-8f1d-16ff45ab4df0 None None] MFA Rules not processed for user `1e37d9ed3751483ebe4141706f46e43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:23.613312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b8942da2-dfc0-4416-8f1d-16ff45ab4df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:23.747363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 977/1931] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:31.732393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=31 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:32.007202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0a449-2478-4380-8aab-03c6ab4b9bba None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:32.007653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0a449-2478-4380-8aab-03c6ab4b9bba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:32.008080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0a449-2478-4380-8aab-03c6ab4b9bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:32.027835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8cb0a449-2478-4380-8aab-03c6ab4b9bba None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:32.029109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8cb0a449-2478-4380-8aab-03c6ab4b9bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:32.036659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 955/1932] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:05:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:32.050262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8942da2-dfc0-4416-8f1d-16ff45ab4df0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:32.062817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8942da2-dfc0-4416-8f1d-16ff45ab4df0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:32.067647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:32.068241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:32.068515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:32.068782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:32.077578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:32.079149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:32.088010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 978/1933] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:05:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 07:05:32.688427 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb529d2-56f5-488b-8d55-70da0b1f200e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:32.688850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb529d2-56f5-488b-8d55-70da0b1f200e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:32.689221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb529d2-56f5-488b-8d55-70da0b1f200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:32.708845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fcb529d2-56f5-488b-8d55-70da0b1f200e None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:32.710029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fcb529d2-56f5-488b-8d55-70da0b1f200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:32.718088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 956/1934] 10.222.0.36 () {58 vars in 967 bytes} [Mon Oct 17 07:05:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:05:32.731815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:32.742149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc31d5d-311a-4923-ab43-108c9cc065d5 service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:32.747810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:32.748498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:32.748849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:32.749189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:32.762622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:32.764343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:32.773547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 979/1935] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:05:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:05:33.043360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2140-a67d-4ccc-a0a2-5b11c9830da2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:33.043899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2140-a67d-4ccc-a0a2-5b11c9830da2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:33.044165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2140-a67d-4ccc-a0a2-5b11c9830da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:33.079709 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4c0a2140-a67d-4ccc-a0a2-5b11c9830da2 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:33.081408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4c0a2140-a67d-4ccc-a0a2-5b11c9830da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:33.096613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 957/1936] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:05:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:33.113414 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:33.124020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d55-4382-479f-99a2-3d46079dbcbc service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:33.129078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:33.129682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:33.129960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:33.130230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:33.139245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:33.140863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:33.150168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 980/1937] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:05:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:05:33.494085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12311c9-efc7-4970-82ed-c5d12e238c3f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:33.494395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12311c9-efc7-4970-82ed-c5d12e238c3f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:33.494690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12311c9-efc7-4970-82ed-c5d12e238c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:33.513316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f12311c9-efc7-4970-82ed-c5d12e238c3f None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:33.514528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f12311c9-efc7-4970-82ed-c5d12e238c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:33.521872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 958/1938] 10.222.0.36 () {58 vars in 967 bytes} [Mon Oct 17 07:05:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:33.534630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:33.545096 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c053-e4c7-4332-afd4-fb0284bd016f service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:33.550258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86b0ac-bba4-4172-a834-4b07af0ad6d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:33.550874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86b0ac-bba4-4172-a834-4b07af0ad6d1 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:33.551151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86b0ac-bba4-4172-a834-4b07af0ad6d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:33.551457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86b0ac-bba4-4172-a834-4b07af0ad6d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:33.560287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86b0ac-bba4-4172-a834-4b07af0ad6d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:33.561847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86b0ac-bba4-4172-a834-4b07af0ad6d1 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:33.569851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 981/1939] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:05:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:05:35.993560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12311c9-efc7-4970-82ed-c5d12e238c3f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:36.071724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12311c9-efc7-4970-82ed-c5d12e238c3f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:36.084957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:36.086638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:36.087107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:36.087509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] PATH_INFO: `/v3/users/7a2a6b0ec599444385bde13a429a7ecb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:36.088841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2a6b0ec599444385bde13a429a7ecb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:36.092999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:36.238127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2a6b0ec599444385bde13a429a7ecb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:36.241354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2a6b0ec599444385bde13a429a7ecb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:36.266075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2a6b0ec599444385bde13a429a7ecb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:36.286924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2a6b0ec599444385bde13a429a7ecb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:36.290239 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2a6b0ec599444385bde13a429a7ecb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:36.322353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 959/1940] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:05:35 2022] DELETE /identity/v3/users/7a2a6b0ec599444385bde13a429a7ecb => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:05:36.327195 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:36.327686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:36.327801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:36.351090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:36.352998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:36.424971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:36.447028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 982/1941] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:36.458107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:36.459686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:36.521921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6e82b-7122-4705-bc47-a746bd9649d7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:36.529076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ec03b-4f28-49f1-bd89-4f004a98b69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:36.529791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec03b-4f28-49f1-bd89-4f004a98b69e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:36.530003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec03b-4f28-49f1-bd89-4f004a98b69e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:36.530257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec03b-4f28-49f1-bd89-4f004a98b69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:36.540233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:36.544501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ec03b-4f28-49f1-bd89-4f004a98b69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:36.547038 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ec03b-4f28-49f1-bd89-4f004a98b69e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:36.563454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 960/1942] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:05:37.214002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:37.215565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:37.227641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae47ab8-8099-48b3-91d1-c5ff530cf390 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:37.233182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:37.234654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.234995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.235637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] PATH_INFO: `/v3/projects/91958c61267b494aabe3e63a13d6f452` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.236969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91958c61267b494aabe3e63a13d6f452)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:37.239938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:37.323443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91958c61267b494aabe3e63a13d6f452', 'request_id': 'req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:37.335090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91958c61267b494aabe3e63a13d6f452', 'request_id': 'req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:37.347924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91958c61267b494aabe3e63a13d6f452', 'request_id': 'req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:37.368973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] The token cache is being invalidate because project 91958c61267b494aabe3e63a13d6f452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:05:37.369646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:37.370302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14f22c24-260b-4acf-a0b0-d3b5e9d7ca74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:05:37.371883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 983/1943] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:05:37 2022] DELETE /identity/v3/projects/91958c61267b494aabe3e63a13d6f452 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:05:37.376913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb904587-94c4-4769-b31e-30dc65b088ab None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.377901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb904587-94c4-4769-b31e-30dc65b088ab None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.378349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb904587-94c4-4769-b31e-30dc65b088ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.403240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eb904587-94c4-4769-b31e-30dc65b088ab None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:37.404509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eb904587-94c4-4769-b31e-30dc65b088ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:37.479232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 961/1944] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:37.490667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1313c6-8538-4b59-a603-a0452f50f5bf None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.491009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1313c6-8538-4b59-a603-a0452f50f5bf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.491237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1313c6-8538-4b59-a603-a0452f50f5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.548288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7a1313c6-8538-4b59-a603-a0452f50f5bf None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:37.552733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7a1313c6-8538-4b59-a603-a0452f50f5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:37.582755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 984/1945] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:37.584107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb904587-94c4-4769-b31e-30dc65b088ab None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:37.597033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb904587-94c4-4769-b31e-30dc65b088ab None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:37.604303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:37.604869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.605146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.605419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.606075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:05:37.606378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:37.608054 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:37.627993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 962/1946] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:05:37.632268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1313c6-8538-4b59-a603-a0452f50f5bf None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:37.643158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1313c6-8538-4b59-a603-a0452f50f5bf None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:37.648333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:37.649840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.650118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.650596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.651441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:37.653555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:37.703275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 985/1947] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:05:37 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:05:37.706167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:37.722045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f8ebe-855c-4f5f-98f8-96cc461c2e9a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:37.793852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:37.794595 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.795004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.795393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.796127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:37.798268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:37.820734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 963/1948] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:05:37 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:05:37.825339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:37.835631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37294fad-206a-41cd-86d6-2d397a22941f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:37.840384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:37.840965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.841238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.841501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.842117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:37.843938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:37.854697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 986/1949] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:05:37.858461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:37.868958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965a949-5cec-445a-a475-0df782cac3ae admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:37.873589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90793278-7ab4-4cb7-b798-4e6614b1be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:37.874177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90793278-7ab4-4cb7-b798-4e6614b1be69 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.874455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90793278-7ab4-4cb7-b798-4e6614b1be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.874728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90793278-7ab4-4cb7-b798-4e6614b1be69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.875396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90793278-7ab4-4cb7-b798-4e6614b1be69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:37.876994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90793278-7ab4-4cb7-b798-4e6614b1be69 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:37.887926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 964/1950] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:05:37.892246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:37.902632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6a2b6-3323-47f6-b6c0-55fb7962232a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:37.907129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:37.907786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.908063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.908335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] PATH_INFO: `/v3/projects/a3f4b1d3bce74b54ac996990fb44d782/users/e65cf0e0271d4dc8adcc2bbb5ad48fb8/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.922708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f4b1d3bce74b54ac996990fb44d782, user_id=e65cf0e0271d4dc8adcc2bbb5ad48fb8, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:37.925385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:37.939165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 987/1951] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:05:37 2022] PUT /identity/v3/projects/a3f4b1d3bce74b54ac996990fb44d782/users/e65cf0e0271d4dc8adcc2bbb5ad48fb8/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:05:37.944459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a0beb-1bd8-414f-bb33-323574dddcec None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:37.944746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a0beb-1bd8-414f-bb33-323574dddcec None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:37.945026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a0beb-1bd8-414f-bb33-323574dddcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:37.966904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d69a0beb-1bd8-414f-bb33-323574dddcec None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:37.968332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d69a0beb-1bd8-414f-bb33-323574dddcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:38.030568 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 965/1952] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:38.040612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:38.042504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:38.110856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef290c-6c0e-436b-912e-f46b0019205f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:38.114823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:38.116051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:38.116051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:38.116281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:38.128262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:38.129824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:38.149324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 988/1953] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:05:39.368599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46216237-0cee-43a0-88de-c24bae0ab130 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:39.369089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46216237-0cee-43a0-88de-c24bae0ab130 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:39.369297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46216237-0cee-43a0-88de-c24bae0ab130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:39.377354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:39.420706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-46216237-0cee-43a0-88de-c24bae0ab130 None None] MFA Rules not processed for user `e65cf0e0271d4dc8adcc2bbb5ad48fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:39.421942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-46216237-0cee-43a0-88de-c24bae0ab130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:39.512722 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 966/1954] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:39.542767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:39.544846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:39.558374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:39.612094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ba2fa-0ea4-4ead-9401-016a4051af1f service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:39.615607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:39.616200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:39.616471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:39.616734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:39.625788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:39.627324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:39.637061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 989/1955] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:05:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:05:41.077181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e57125-af52-42f3-b0af-600eb6aaed98 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:41.077651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e57125-af52-42f3-b0af-600eb6aaed98 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:41.077651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e57125-af52-42f3-b0af-600eb6aaed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:41.095993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-37e57125-af52-42f3-b0af-600eb6aaed98 None None] MFA Rules not processed for user `e65cf0e0271d4dc8adcc2bbb5ad48fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:05:41.097165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-37e57125-af52-42f3-b0af-600eb6aaed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:41.104199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 967/1956] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:05:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:05:49.632347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=23 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:51.091190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=34 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:05:53.375248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:05:53.378161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:53.398109 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88647-bb5b-4c28-9723-7a8f78357283 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:05:53.407103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:05:53.408280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:05:53.408280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:05:53.408496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:05:53.410719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:05:53.494717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:05:53.496192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:05:53.508013 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 990/1957] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:05:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:06:03.502401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=30 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:06:03.761906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b582a80-0b32-4891-b9e6-53e03ab3165a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:03.762387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b582a80-0b32-4891-b9e6-53e03ab3165a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:03.762499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b582a80-0b32-4891-b9e6-53e03ab3165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:03.781356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8b582a80-0b32-4891-b9e6-53e03ab3165a None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:03.782530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8b582a80-0b32-4891-b9e6-53e03ab3165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:03.836819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 968/1958] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:06:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:06:03.858607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:03.860019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:03.927770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1a52d-a22b-487b-9d65-1f89c7a6ed39 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:03.933626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:03.934226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:03.934630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:03.934747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:03.943241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:03.944996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:03.959864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 991/1959] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:06:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:06:06.479375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:06.479930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:06.479930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:06.512859 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:06.514092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:06.522548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 969/1960] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:06:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:06:06.538507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:06.550974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687f5c7-32f1-4594-b77a-802c04e0a5c1 service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:06.557838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2022a5-e2b5-4c42-add3-c08b85faf8f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:06.558452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2022a5-e2b5-4c42-add3-c08b85faf8f7 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:06.558725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2022a5-e2b5-4c42-add3-c08b85faf8f7 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:06.559007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2022a5-e2b5-4c42-add3-c08b85faf8f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:06.567951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2022a5-e2b5-4c42-add3-c08b85faf8f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:06.569640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2022a5-e2b5-4c42-add3-c08b85faf8f7 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:06.578532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 992/1961] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:06:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:06:10.460375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:10.463179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:10.604570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839174cc-bce1-4d38-b3c6-c8eb31831324 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:10.612618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa9d26-c469-4cd4-a58a-10af7d8deec4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:10.613382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa9d26-c469-4cd4-a58a-10af7d8deec4 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:10.613759 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa9d26-c469-4cd4-a58a-10af7d8deec4 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:10.614124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa9d26-c469-4cd4-a58a-10af7d8deec4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:10.616299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a2aa9d26-c469-4cd4-a58a-10af7d8deec4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:10.736823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa9d26-c469-4cd4-a58a-10af7d8deec4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:10.745102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa9d26-c469-4cd4-a58a-10af7d8deec4 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:10.758876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 970/1962] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:06:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:06:16.574732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=38 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:06:20.753107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=60 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:06:22.764383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be2f1f-6bb4-4895-8591-df456c1c2db4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:22.764964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be2f1f-6bb4-4895-8591-df456c1c2db4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:22.764964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be2f1f-6bb4-4895-8591-df456c1c2db4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:22.766260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 993/1963] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 07:06:22.784672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738b0e1-ba06-4ca5-b253-289872078039 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:22.785302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738b0e1-ba06-4ca5-b253-289872078039 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:22.785302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738b0e1-ba06-4ca5-b253-289872078039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:22.833580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f738b0e1-ba06-4ca5-b253-289872078039 None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:22.835382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f738b0e1-ba06-4ca5-b253-289872078039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:22.918447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 971/1964] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:06:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:06:23.145373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be2f1f-6bb4-4895-8591-df456c1c2db4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:23.151229 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-81be2f1f-6bb4-4895-8591-df456c1c2db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:23.235544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be2f1f-6bb4-4895-8591-df456c1c2db4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:23.239067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3efa31-7f05-452c-b46b-f8f5e0611a83 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:23.239696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3efa31-7f05-452c-b46b-f8f5e0611a83 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:23.239978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3efa31-7f05-452c-b46b-f8f5e0611a83 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:23.240252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3efa31-7f05-452c-b46b-f8f5e0611a83 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:23.250276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3efa31-7f05-452c-b46b-f8f5e0611a83 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:23.251731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3efa31-7f05-452c-b46b-f8f5e0611a83 service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:23.262786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 994/1965] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:06:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:06:23.346903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b14a-55c2-4e8f-aaa1-56b4fc13977e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:23.347235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b14a-55c2-4e8f-aaa1-56b4fc13977e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:23.347528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b14a-55c2-4e8f-aaa1-56b4fc13977e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:23.348869 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 972/1966] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:06:23.353548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6318-80da-4961-b063-40ffbe928ffa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:23.353856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6318-80da-4961-b063-40ffbe928ffa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:23.354149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6318-80da-4961-b063-40ffbe928ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:23.374038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9a9a6318-80da-4961-b063-40ffbe928ffa None None] MFA Rules not processed for user `56967a5531ea4e308af5b89c13524c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:23.375488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9a9a6318-80da-4961-b063-40ffbe928ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:23.381946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 995/1967] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:06:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:06:23.394165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b14a-55c2-4e8f-aaa1-56b4fc13977e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:23.405068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b14a-55c2-4e8f-aaa1-56b4fc13977e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:23.408620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab77db-c729-4c07-a1d7-2aadc9912b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:23.409193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab77db-c729-4c07-a1d7-2aadc9912b72 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:23.409471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab77db-c729-4c07-a1d7-2aadc9912b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:23.409740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab77db-c729-4c07-a1d7-2aadc9912b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:23.416703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab77db-c729-4c07-a1d7-2aadc9912b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:23.418128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab77db-c729-4c07-a1d7-2aadc9912b72 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:23.425842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 973/1968] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:06:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:06:23.508501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe2477-60fa-4b2b-8026-01cad4a65f21 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:23.508872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe2477-60fa-4b2b-8026-01cad4a65f21 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:23.509168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe2477-60fa-4b2b-8026-01cad4a65f21 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:23.510654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 996/1969] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:06:23.515881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47e32b-4d6b-43a1-8fc1-e747adfe34ab None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:23.516168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47e32b-4d6b-43a1-8fc1-e747adfe34ab None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:23.516443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47e32b-4d6b-43a1-8fc1-e747adfe34ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:23.536422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5b47e32b-4d6b-43a1-8fc1-e747adfe34ab None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:23.537622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b47e32b-4d6b-43a1-8fc1-e747adfe34ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:23.547289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 974/1970] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:06:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:06:23.557604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe2477-60fa-4b2b-8026-01cad4a65f21 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:23.569124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe2477-60fa-4b2b-8026-01cad4a65f21 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:23.572726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:23.573339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:23.573646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:23.573913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:23.584619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:23.586037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:23.597844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 997/1971] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:06:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:06:24.322436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:24.322913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:24.323015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:24.342998 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:24.344161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:24.351993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 975/1972] 10.222.0.36 () {58 vars in 967 bytes} [Mon Oct 17 07:06:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:06:24.364582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:24.378087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bacca-352a-44ec-adbb-5de307e309ee service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:24.383464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:24.384641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:24.384912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:24.385177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:24.392936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:24.394507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:24.402411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 998/1973] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:06:26.378383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:26.380041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:26.396085 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e5d16-cecf-409f-a124-46aa6ed39e55 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:26.405845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:26.406779 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:26.407350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:26.412488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] PATH_INFO: `/v3/users/2e07169459e44b079cbcc5157874c89a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:26.414870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e07169459e44b079cbcc5157874c89a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:26.417783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:26.476681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e07169459e44b079cbcc5157874c89a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.477501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e07169459e44b079cbcc5157874c89a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.485862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e07169459e44b079cbcc5157874c89a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.500259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e07169459e44b079cbcc5157874c89a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.500804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e38257cc-8276-473d-9a81-1b8595302f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e07169459e44b079cbcc5157874c89a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.521182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 976/1974] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:06:26 2022] DELETE /identity/v3/users/2e07169459e44b079cbcc5157874c89a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:06:26.764722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:26.779052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d9fe7-3cf7-4b6a-af38-59697ec75ef7 service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:26.849862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:26.850469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:26.850745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:26.851066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] PATH_INFO: `/v3/projects/58fdf05cfb044a2aa98bc12a38fb76cd` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:26.852162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58fdf05cfb044a2aa98bc12a38fb76cd)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:26.854579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:26.904890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58fdf05cfb044a2aa98bc12a38fb76cd', 'request_id': 'req-b7638850-ddfa-4a92-9b00-414fb3004c8d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.910320 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58fdf05cfb044a2aa98bc12a38fb76cd', 'request_id': 'req-b7638850-ddfa-4a92-9b00-414fb3004c8d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.917343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58fdf05cfb044a2aa98bc12a38fb76cd', 'request_id': 'req-b7638850-ddfa-4a92-9b00-414fb3004c8d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.928170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:06:26.933374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] The token cache is being invalidate because project 58fdf05cfb044a2aa98bc12a38fb76cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:06:26.933779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.934815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b7638850-ddfa-4a92-9b00-414fb3004c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:06:26.936527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 999/1975] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:06:26 2022] DELETE /identity/v3/projects/58fdf05cfb044a2aa98bc12a38fb76cd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:06:26.941897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a878b9-8e08-44a4-8c7c-b1411d2eb2da None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:26.942177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a878b9-8e08-44a4-8c7c-b1411d2eb2da None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:26.942450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a878b9-8e08-44a4-8c7c-b1411d2eb2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:26.963595 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c5a878b9-8e08-44a4-8c7c-b1411d2eb2da None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:26.964841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c5a878b9-8e08-44a4-8c7c-b1411d2eb2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:27.010053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:06:27.040679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 977/1976] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:06:27.048409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b887a9-c3df-472f-a6b7-a844726f27d0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.048694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b887a9-c3df-472f-a6b7-a844726f27d0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.049160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b887a9-c3df-472f-a6b7-a844726f27d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.070137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-03b887a9-c3df-472f-a6b7-a844726f27d0 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:27.071974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-03b887a9-c3df-472f-a6b7-a844726f27d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:27.080270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1000/1977] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:06:27.084514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a878b9-8e08-44a4-8c7c-b1411d2eb2da None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:27.094274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a878b9-8e08-44a4-8c7c-b1411d2eb2da None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:27.099352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:27.100048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.100358 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.100729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.101475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:06:27.101859 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:27.103860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:27.116342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 978/1978] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:06:27.121412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b887a9-c3df-472f-a6b7-a844726f27d0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:27.132032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b887a9-c3df-472f-a6b7-a844726f27d0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:27.136981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:27.137568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.137848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.138123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.138762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:27.140711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:27.178434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1001/1979] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:06:27 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:06:27.179500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:27.189734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d308c9-0eff-4905-9ee5-8c11b72ad7b7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:27.247821 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:27.248423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.248698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.248965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.249584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:27.251560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:27.273045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 979/1980] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:06:27 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:06:27.278144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:27.288513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a49d0-fc22-4091-b1ee-6d76399204b9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:27.294004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:27.294599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.294877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.295149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.295815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:27.297398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:27.307283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1002/1981] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:06:27.312407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:27.322542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55772cba-061b-4e12-b3ea-6c6dbc8c2f9f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:27.327442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f956c-3209-4034-87de-91225817a48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:27.328036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f956c-3209-4034-87de-91225817a48e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.328311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f956c-3209-4034-87de-91225817a48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.328581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f956c-3209-4034-87de-91225817a48e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.329208 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f956c-3209-4034-87de-91225817a48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:27.330805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f956c-3209-4034-87de-91225817a48e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:27.344764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 980/1982] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:06:27.350517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:27.363651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3961e5-65d4-4d1a-a2ee-033bffeea415 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:27.368913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17f72c-3d39-4ee3-93c3-38fcde643162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:27.369617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f72c-3d39-4ee3-93c3-38fcde643162 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.369898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f72c-3d39-4ee3-93c3-38fcde643162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.370203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f72c-3d39-4ee3-93c3-38fcde643162 admin admin] PATH_INFO: `/v3/projects/6e751f22a91f45fc90fc6b8540d96884/users/47f5aed503454facbc7b54eb5fb9c606/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.385094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17f72c-3d39-4ee3-93c3-38fcde643162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e751f22a91f45fc90fc6b8540d96884, user_id=47f5aed503454facbc7b54eb5fb9c606, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:27.387934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17f72c-3d39-4ee3-93c3-38fcde643162 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:27.401606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1003/1983] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:06:27 2022] PUT /identity/v3/projects/6e751f22a91f45fc90fc6b8540d96884/users/47f5aed503454facbc7b54eb5fb9c606/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:06:27.411641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38761465-be69-4825-b691-a42565a0e28e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.412145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38761465-be69-4825-b691-a42565a0e28e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.412273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38761465-be69-4825-b691-a42565a0e28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.427152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.427473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.427754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.442588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-38761465-be69-4825-b691-a42565a0e28e None None] MFA Rules not processed for user `47f5aed503454facbc7b54eb5fb9c606`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:27.443831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-38761465-be69-4825-b691-a42565a0e28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:27.447649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:06:27.448818 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:27.507892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1004/1984] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:06:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:06:27.511174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 981/1985] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:06:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:06:27.521065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:27.522404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:27.590701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a02a4-8cbc-4bc3-84c3-ace78a648ea4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:27.595700 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c5727-8db6-4cf8-af50-fd4a1147b79e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:27.596296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c5727-8db6-4cf8-af50-fd4a1147b79e service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:27.596602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c5727-8db6-4cf8-af50-fd4a1147b79e service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:27.596867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c5727-8db6-4cf8-af50-fd4a1147b79e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:27.605119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c5727-8db6-4cf8-af50-fd4a1147b79e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:27.606688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c5727-8db6-4cf8-af50-fd4a1147b79e service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:27.614521 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1005/1986] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:06:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 07:06:34.272243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38761465-be69-4825-b691-a42565a0e28e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:06:34.273729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-38761465-be69-4825-b691-a42565a0e28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:06:34.321524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:06:34.345820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38761465-be69-4825-b691-a42565a0e28e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:06:34.349768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:06:34.350370 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:06:34.350641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:06:34.350910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:06:34.359422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:06:34.360846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:06:34.369454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 982/1987] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:06:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:06:37.611516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:06:44.365571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=11 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:06.664580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5f2c-7d7f-44e1-bc55-ed9bf49a9649 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:06.665074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5f2c-7d7f-44e1-bc55-ed9bf49a9649 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:06.665172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5f2c-7d7f-44e1-bc55-ed9bf49a9649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:06.683968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c22b5f2c-7d7f-44e1-bc55-ed9bf49a9649 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:06.685157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c22b5f2c-7d7f-44e1-bc55-ed9bf49a9649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:06.692193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1006/1988] 10.222.0.36 () {58 vars in 967 bytes} [Mon Oct 17 07:07:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:06.714458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:06.725879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d37734-b24d-4a4c-979e-0f6a79507ab0 service glance] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:06.730777 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53931fa3-2131-4d3e-823e-a60134b8361b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:06.731404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53931fa3-2131-4d3e-823e-a60134b8361b service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:06.731682 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53931fa3-2131-4d3e-823e-a60134b8361b service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:06.731952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53931fa3-2131-4d3e-823e-a60134b8361b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:06.739711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53931fa3-2131-4d3e-823e-a60134b8361b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:06.741278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53931fa3-2131-4d3e-823e-a60134b8361b service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:06.749765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 983/1989] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:07:16.679000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:16.746554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:23.923648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b5f2c-7d7f-44e1-bc55-ed9bf49a9649 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:23.984987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b5f2c-7d7f-44e1-bc55-ed9bf49a9649 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:23.989406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:23.990063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:23.990341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:23.990606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] PATH_INFO: `/v3/users/47f5aed503454facbc7b54eb5fb9c606` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:23.991735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f5aed503454facbc7b54eb5fb9c606)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:23.993648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:24.039393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f5aed503454facbc7b54eb5fb9c606'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.040035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f5aed503454facbc7b54eb5fb9c606'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.047440 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f5aed503454facbc7b54eb5fb9c606'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.048139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f5aed503454facbc7b54eb5fb9c606'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.061022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f5aed503454facbc7b54eb5fb9c606'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.073230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1007/1990] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:07:23 2022] DELETE /identity/v3/users/47f5aed503454facbc7b54eb5fb9c606 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:07:24.078562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:24.079001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:24.079188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:24.097967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:24.099160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:24.169077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 984/1991] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:24.181097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:24.183235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:24.247996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f5174-6d1a-4adf-b240-c5b987626df4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:24.253132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff436220-696f-4b5a-96d2-447a93c20a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:24.253795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436220-696f-4b5a-96d2-447a93c20a12 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:24.254092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436220-696f-4b5a-96d2-447a93c20a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:24.254363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436220-696f-4b5a-96d2-447a93c20a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:24.265035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff436220-696f-4b5a-96d2-447a93c20a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:24.266539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff436220-696f-4b5a-96d2-447a93c20a12 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:24.277808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1008/1992] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:07:24.745185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:24.746636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:24.758696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8032a33-095d-4b47-ac35-ef81c727f5de None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:24.763390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:24.764022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:24.764296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:24.764593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] PATH_INFO: `/v3/projects/6e751f22a91f45fc90fc6b8540d96884` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:24.765702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e751f22a91f45fc90fc6b8540d96884)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:24.767696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:24.815734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e751f22a91f45fc90fc6b8540d96884', 'request_id': 'req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.821225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e751f22a91f45fc90fc6b8540d96884', 'request_id': 'req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.829129 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e751f22a91f45fc90fc6b8540d96884', 'request_id': 'req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.840725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] The token cache is being invalidate because project 6e751f22a91f45fc90fc6b8540d96884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:07:24.841005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.841623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f83e2ce7-c6cd-4c56-8490-cc0922e32f4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:24.843133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 985/1993] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:07:24 2022] DELETE /identity/v3/projects/6e751f22a91f45fc90fc6b8540d96884 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:07:24.848246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f130de-6c8f-42f3-8e16-fb5770980969 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:24.848531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f130de-6c8f-42f3-8e16-fb5770980969 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:24.848805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f130de-6c8f-42f3-8e16-fb5770980969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:24.868410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-62f130de-6c8f-42f3-8e16-fb5770980969 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:24.869592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-62f130de-6c8f-42f3-8e16-fb5770980969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:24.885863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:24.929403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1009/1994] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:24.937873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cb20c-d4fa-47e2-b0a1-65f857e66d50 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:24.938158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cb20c-d4fa-47e2-b0a1-65f857e66d50 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:24.938429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cb20c-d4fa-47e2-b0a1-65f857e66d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:24.959501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-180cb20c-d4fa-47e2-b0a1-65f857e66d50 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:24.960687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-180cb20c-d4fa-47e2-b0a1-65f857e66d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:24.968431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 986/1995] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:24.972581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f130de-6c8f-42f3-8e16-fb5770980969 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:24.981989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f130de-6c8f-42f3-8e16-fb5770980969 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:24.986422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:24.987008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:24.987309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:24.988006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:24.988638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:07:24.988935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:24.990502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.002228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1010/1996] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:07:25.006668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cb20c-d4fa-47e2-b0a1-65f857e66d50 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.016478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cb20c-d4fa-47e2-b0a1-65f857e66d50 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.021339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.021971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.022250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.022523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.023144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.025123 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.051071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 987/1997] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:07:25 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:07:25.055273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.068248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38291005-db3b-47b8-93dc-2975f651bf69 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.131858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.132461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.132745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.133013 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.133626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.135579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.163557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1011/1998] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:07:25 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:07:25.168194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.178634 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dc0e9-30ad-43d5-8bfd-02dece468baa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.183471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.184039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.184341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.184612 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.185242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.186876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.197223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 988/1999] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:07:25.201754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.211520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5672678-8ac1-4ca3-bf80-880a87ad147d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.216665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d33ab-5a5d-40a4-97dd-483119882ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.217250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d33ab-5a5d-40a4-97dd-483119882ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.217526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d33ab-5a5d-40a4-97dd-483119882ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.217799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d33ab-5a5d-40a4-97dd-483119882ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.218453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d33ab-5a5d-40a4-97dd-483119882ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.220067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d33ab-5a5d-40a4-97dd-483119882ca7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.231635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1012/2000] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:07:25.235828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.246033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05964a64-8716-4768-acb1-afaa1b9bb2d9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.251000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.251855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.252133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.252428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] PATH_INFO: `/v3/projects/77a89265c40641ce97dce2f574d08214/users/c996d789bd414cf2808ffc2213fe4325/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.269132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a89265c40641ce97dce2f574d08214, user_id=c996d789bd414cf2808ffc2213fe4325, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.272166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.286271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 989/2001] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:07:25 2022] PUT /identity/v3/projects/77a89265c40641ce97dce2f574d08214/users/c996d789bd414cf2808ffc2213fe4325/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:07:25.297561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3e6b-759a-41bb-ba0f-6b2fc1ce3d7a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.297846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3e6b-759a-41bb-ba0f-6b2fc1ce3d7a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.298123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3e6b-759a-41bb-ba0f-6b2fc1ce3d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.326232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-efaf3e6b-759a-41bb-ba0f-6b2fc1ce3d7a None None] MFA Rules not processed for user `c996d789bd414cf2808ffc2213fe4325`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:25.327481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-efaf3e6b-759a-41bb-ba0f-6b2fc1ce3d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:25.397197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1013/2002] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:25.409383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.411047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:25.432202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:25.469841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de462744-737c-4541-a110-57bc29d88d93 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.473411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754a656-a821-4a71-b2cd-25acae218d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.474002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754a656-a821-4a71-b2cd-25acae218d03 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.474276 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754a656-a821-4a71-b2cd-25acae218d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.474545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754a656-a821-4a71-b2cd-25acae218d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.482585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754a656-a821-4a71-b2cd-25acae218d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.484032 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754a656-a821-4a71-b2cd-25acae218d03 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.492586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 990/2003] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:07:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:07:25.518409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3e6b-759a-41bb-ba0f-6b2fc1ce3d7a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.537305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:25.584361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3e6b-759a-41bb-ba0f-6b2fc1ce3d7a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.590466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.591103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.591413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.591684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] PATH_INFO: `/v3/users/c996d789bd414cf2808ffc2213fe4325` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.592810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c996d789bd414cf2808ffc2213fe4325)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.594743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.645688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c996d789bd414cf2808ffc2213fe4325'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:25.646422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c996d789bd414cf2808ffc2213fe4325'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:25.653893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c996d789bd414cf2808ffc2213fe4325'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:25.654631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c996d789bd414cf2808ffc2213fe4325'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:25.668932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c996d789bd414cf2808ffc2213fe4325'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:25.682209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1014/2004] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:07:25 2022] DELETE /identity/v3/users/c996d789bd414cf2808ffc2213fe4325 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:07:25.687295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.687478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.687749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.707440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:25.708661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:25.795445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 991/2005] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:25.796047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:25.796047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:25.854182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850ced9-7ba5-4c5e-924d-2c0528ec2ccd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:25.858644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238ffa1-cbe3-4a36-882d-ee70a7f69ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:25.859425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ffa1-cbe3-4a36-882d-ee70a7f69ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:25.859699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ffa1-cbe3-4a36-882d-ee70a7f69ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:25.859965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ffa1-cbe3-4a36-882d-ee70a7f69ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:25.869159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238ffa1-cbe3-4a36-882d-ee70a7f69ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:25.870654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238ffa1-cbe3-4a36-882d-ee70a7f69ed5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:25.887040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1015/2006] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:07:26.149733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:26.160600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:26.174193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851b566-22b8-44c9-89a0-51bb9892b3cb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:26.179004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:26.179658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:26.179929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:26.180193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] PATH_INFO: `/v3/projects/77a89265c40641ce97dce2f574d08214` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:26.181286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a89265c40641ce97dce2f574d08214)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:26.183799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:26.228931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a89265c40641ce97dce2f574d08214', 'request_id': 'req-3db413ea-6427-4208-93f1-8c849d185084'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:26.234942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a89265c40641ce97dce2f574d08214', 'request_id': 'req-3db413ea-6427-4208-93f1-8c849d185084'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:26.241805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a89265c40641ce97dce2f574d08214', 'request_id': 'req-3db413ea-6427-4208-93f1-8c849d185084'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:26.252836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] The token cache is being invalidate because project 77a89265c40641ce97dce2f574d08214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:07:26.253114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:26.253729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3db413ea-6427-4208-93f1-8c849d185084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:07:26.255187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 992/2007] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:07:26 2022] DELETE /identity/v3/projects/77a89265c40641ce97dce2f574d08214 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:07:32.952349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef21e-59d2-44eb-a989-e6de5f9acf37 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:32.952880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef21e-59d2-44eb-a989-e6de5f9acf37 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:32.952880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef21e-59d2-44eb-a989-e6de5f9acf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:32.972532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-128ef21e-59d2-44eb-a989-e6de5f9acf37 None None] MFA Rules not processed for user `1e37d9ed3751483ebe4141706f46e43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:32.973669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-128ef21e-59d2-44eb-a989-e6de5f9acf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:33.008136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:33.032696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1016/2008] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:07:36.249973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:39.319906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:39.320496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:39.320496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:39.341826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] MFA Rules not processed for user `e65cf0e0271d4dc8adcc2bbb5ad48fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:39.343022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:39.404330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 993/2009] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:43.024736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:47.692835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:47.693372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:47.693461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:47.712322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] MFA Rules not processed for user `c987a5a52fc641f78054ac0218edcd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:07:47.713556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:47.772664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1017/2010] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:07:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:07:49.397328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=25 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:57.475533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:57.477046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:57.532966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c26d5-a139-4b0b-bdb2-a967067b8983 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:57.536528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe601d73-63d3-4de4-b68c-a8dc69423007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:57.537127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe601d73-63d3-4de4-b68c-a8dc69423007 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:57.537404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe601d73-63d3-4de4-b68c-a8dc69423007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:57.537679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe601d73-63d3-4de4-b68c-a8dc69423007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:57.539410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fe601d73-63d3-4de4-b68c-a8dc69423007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:57.609424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe601d73-63d3-4de4-b68c-a8dc69423007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:57.610866 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe601d73-63d3-4de4-b68c-a8dc69423007 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:57.622158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 994/2011] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:07:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:07:57.765707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=25 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:07:57.799168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:07:57.800595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:57.860822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058cc4f-9359-4080-9a59-d438b00af13c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:07:57.864361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:07:57.864967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:07:57.865244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:07:57.865515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:07:57.867511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:07:57.882483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:07:57.884241 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:07:57.895496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1018/2012] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:07:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:07.616891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=54 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:07.890348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=26 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:21.259421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:21.260452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:21.260565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:21.280695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] MFA Rules not processed for user `8d404b2974f64aa8a6c32adfc3f7b0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:08:21.281882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:21.335569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 995/2013] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:08:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:08:21.635070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:21.644247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9ed7f-c89c-4262-9b0d-3d7aa08ea609 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:21.647837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:21.648428 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:21.648701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:21.648969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:21.651235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:21.667975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:21.669438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:21.677842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1019/2014] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:22.322172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:22.324200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:22.398752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b150e5a-ca09-4a49-917c-83d3a7ccc851 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:22.403603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9954e-7ba0-4c06-a134-14cd83ebcab6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:22.404201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9954e-7ba0-4c06-a134-14cd83ebcab6 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:22.404473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9954e-7ba0-4c06-a134-14cd83ebcab6 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:22.404739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9954e-7ba0-4c06-a134-14cd83ebcab6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:22.406420 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-78f9954e-7ba0-4c06-a134-14cd83ebcab6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:22.417314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9954e-7ba0-4c06-a134-14cd83ebcab6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:22.418873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9954e-7ba0-4c06-a134-14cd83ebcab6 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:22.428368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 996/2015] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:25.528639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:25.539523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5960e-fdc2-495b-862e-b8361fce5807 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:25.544849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:25.545896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:25.546143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:25.546564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:25.548545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:25.561322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:25.563121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:25.576540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1020/2016] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:25.739547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76a56-b87a-4eda-833c-c28ed71024b9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:25.739999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76a56-b87a-4eda-833c-c28ed71024b9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:25.740284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76a56-b87a-4eda-833c-c28ed71024b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:25.761923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c8d76a56-b87a-4eda-833c-c28ed71024b9 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:08:25.763153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8d76a56-b87a-4eda-833c-c28ed71024b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:25.819046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 997/2017] 10.222.0.36 () {58 vars in 967 bytes} [Mon Oct 17 07:08:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:08:25.868207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:25.870170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:25.952517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043698-9682-4fa3-9c48-46b4867100ee service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:25.957692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:25.958370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:25.958718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:25.958987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:25.967193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:25.968844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:25.977326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1021/2018] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:08:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:08:29.790167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76a56-b87a-4eda-833c-c28ed71024b9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:29.802299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76a56-b87a-4eda-833c-c28ed71024b9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:29.807897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:29.808501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:29.808781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:29.809053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:29.810755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:29.829380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:29.831336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:29.845839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 998/2019] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:33.502569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:33.504270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:33.574173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0145fd-f8c2-4dfd-b185-0f2dc19b925c service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:33.578797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:33.579470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:33.579742 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:33.580008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] PATH_INFO: `/v3/users/c09a31533d7847fea61682ff032f838b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:33.581088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c09a31533d7847fea61682ff032f838b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:33.583091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:33.629673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c09a31533d7847fea61682ff032f838b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:33.630457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c09a31533d7847fea61682ff032f838b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:33.637894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c09a31533d7847fea61682ff032f838b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:33.638545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c09a31533d7847fea61682ff032f838b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:33.645497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:33.651743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c09a31533d7847fea61682ff032f838b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:33.663909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1022/2020] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:08:33 2022] DELETE /identity/v3/users/c09a31533d7847fea61682ff032f838b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:08:33.846289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:33.861518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd48fb2-15f8-479b-95db-9119efe63291 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:33.890591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:33.925848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:33.926473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:33.926751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:33.927020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] PATH_INFO: `/v3/projects/db8098f6f78c49fa843d34d69b002daa` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:33.928148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8098f6f78c49fa843d34d69b002daa)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:33.930120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:33.983236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8098f6f78c49fa843d34d69b002daa', 'request_id': 'req-7ee6a352-a700-4f5c-9f5a-1991c1784e61'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:33.988963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8098f6f78c49fa843d34d69b002daa', 'request_id': 'req-7ee6a352-a700-4f5c-9f5a-1991c1784e61'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:33.998767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8098f6f78c49fa843d34d69b002daa', 'request_id': 'req-7ee6a352-a700-4f5c-9f5a-1991c1784e61'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.013784 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] The token cache is being invalidate because project db8098f6f78c49fa843d34d69b002daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:08:34.014236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.015018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.016669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 999/2021] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:08:33 2022] DELETE /identity/v3/projects/db8098f6f78c49fa843d34d69b002daa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:08:34.020692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:34.022185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:34.035946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d5ba6-fc39-432f-8bed-d688a85da417 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:34.161144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:34.161920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:34.163325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:34.163677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] PATH_INFO: `/v3/users/62038532ab4e4398ab43e593c7ec1224` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:34.165244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62038532ab4e4398ab43e593c7ec1224)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:34.167855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:34.219951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62038532ab4e4398ab43e593c7ec1224'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.220621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62038532ab4e4398ab43e593c7ec1224'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.227500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62038532ab4e4398ab43e593c7ec1224'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.228181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62038532ab4e4398ab43e593c7ec1224'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.240855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62038532ab4e4398ab43e593c7ec1224'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.253387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1023/2022] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:08:34 2022] DELETE /identity/v3/users/62038532ab4e4398ab43e593c7ec1224 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:08:34.437995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:34.439552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:34.454321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6a352-a700-4f5c-9f5a-1991c1784e61 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:34.534980 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:34.535700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:34.536122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:34.536445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] PATH_INFO: `/v3/projects/a5c5bdd7cef34826843be339ab43e251` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:34.537721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c5bdd7cef34826843be339ab43e251)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:34.539775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:34.594817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c5bdd7cef34826843be339ab43e251', 'request_id': 'req-b9ba141e-115b-4e2a-8579-f172252c52fe'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.600215 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c5bdd7cef34826843be339ab43e251', 'request_id': 'req-b9ba141e-115b-4e2a-8579-f172252c52fe'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.601931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:34.609665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c5bdd7cef34826843be339ab43e251', 'request_id': 'req-b9ba141e-115b-4e2a-8579-f172252c52fe'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.623289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] The token cache is being invalidate because project a5c5bdd7cef34826843be339ab43e251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:08:34.623578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.624639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:34.626173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1000/2023] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:08:34 2022] DELETE /identity/v3/projects/a5c5bdd7cef34826843be339ab43e251 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:08:35.337131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:35.339461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:35.399157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d12d5b-7031-4bc7-8789-fc65a7a10e43 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:35.411352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:35.448589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:35.449192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:35.449465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:35.449732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:35.451709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:35.507702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:35.509130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:35.518477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1024/2024] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:38.128563 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:38.130094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:38.205273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba141e-115b-4e2a-8579-f172252c52fe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:38.210747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:38.211405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:38.211673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:38.211949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:38.213703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:38.274133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:38.275807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:38.286055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1001/2025] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:43.038593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:43.052189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1ce5a-35be-4a8c-a876-ed8caabe9f67 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:43.058784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:43.059896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:43.060399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:43.060885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:43.063104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:43.077677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:43.079813 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:43.097833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1025/2026] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:43.210600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:43.226332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00c3b-b26c-4d3d-b378-cb42552b2a77 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:43.255264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d8925-eb1e-43f0-a4a2-6f87e1930f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:43.256236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d8925-eb1e-43f0-a4a2-6f87e1930f45 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:43.256688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d8925-eb1e-43f0-a4a2-6f87e1930f45 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:43.257123 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d8925-eb1e-43f0-a4a2-6f87e1930f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:43.259828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-944d8925-eb1e-43f0-a4a2-6f87e1930f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:43.294618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d8925-eb1e-43f0-a4a2-6f87e1930f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:43.296250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d8925-eb1e-43f0-a4a2-6f87e1930f45 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:43.334283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1002/2027] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:44.966689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:44.968129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:44.979542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d779b9-76c5-4158-88ef-75e505fd3364 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:44.983290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:44.983963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:44.984247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:44.984698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:44.986235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:45.049987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:45.051687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:45.065180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1026/2028] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:08:46.359505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547941d7-5a95-4a63-a2fc-1e7850b8f528 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:46.359982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547941d7-5a95-4a63-a2fc-1e7850b8f528 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:46.360093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547941d7-5a95-4a63-a2fc-1e7850b8f528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:46.380584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-547941d7-5a95-4a63-a2fc-1e7850b8f528 None None] MFA Rules not processed for user `51b2f292cea24a21928e2adcef0c3786`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:08:46.381823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-547941d7-5a95-4a63-a2fc-1e7850b8f528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:46.390425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1003/2029] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:08:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:08:49.763441 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:49.770130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:49.849600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:49.855834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ec532-ac5d-49a2-b576-87954b47349d service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:49.865957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:49.866848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:49.867321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:49.867763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] PATH_INFO: `/v3/users/e65cf0e0271d4dc8adcc2bbb5ad48fb8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:49.869205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65cf0e0271d4dc8adcc2bbb5ad48fb8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:49.871526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:49.938689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65cf0e0271d4dc8adcc2bbb5ad48fb8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:49.939596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65cf0e0271d4dc8adcc2bbb5ad48fb8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:49.955586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65cf0e0271d4dc8adcc2bbb5ad48fb8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:49.956329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65cf0e0271d4dc8adcc2bbb5ad48fb8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:49.970650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9805a5a9-7ba4-4264-8fe7-2b50eaafb39a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65cf0e0271d4dc8adcc2bbb5ad48fb8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:49.996653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1027/2030] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:08:49 2022] DELETE /identity/v3/users/e65cf0e0271d4dc8adcc2bbb5ad48fb8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:08:50.193412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547941d7-5a95-4a63-a2fc-1e7850b8f528 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:50.221559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547941d7-5a95-4a63-a2fc-1e7850b8f528 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:50.284466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:50.285104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.285372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.285637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] PATH_INFO: `/v3/projects/a3f4b1d3bce74b54ac996990fb44d782` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.286775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f4b1d3bce74b54ac996990fb44d782)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:50.288965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:50.296228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:50.357018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f4b1d3bce74b54ac996990fb44d782', 'request_id': 'req-2e2235e1-2717-462a-a37a-bf4686c1e20e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:50.363906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f4b1d3bce74b54ac996990fb44d782', 'request_id': 'req-2e2235e1-2717-462a-a37a-bf4686c1e20e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:50.372780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f4b1d3bce74b54ac996990fb44d782', 'request_id': 'req-2e2235e1-2717-462a-a37a-bf4686c1e20e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:50.387803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] The token cache is being invalidate because project a3f4b1d3bce74b54ac996990fb44d782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:08:50.388091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:50.388753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e2235e1-2717-462a-a37a-bf4686c1e20e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:08:50.390225 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1004/2031] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:08:50 2022] DELETE /identity/v3/projects/a3f4b1d3bce74b54ac996990fb44d782 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:08:50.395744 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29311358-bc7b-4a52-b62e-e3570b70ba1a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.396083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29311358-bc7b-4a52-b62e-e3570b70ba1a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.396325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29311358-bc7b-4a52-b62e-e3570b70ba1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.419489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-29311358-bc7b-4a52-b62e-e3570b70ba1a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:08:50.420620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-29311358-bc7b-4a52-b62e-e3570b70ba1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:50.487478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1028/2032] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:08:50.495467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa85a7-2c11-4e52-aa70-550b7a211d94 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.495853 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa85a7-2c11-4e52-aa70-550b7a211d94 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.496214 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa85a7-2c11-4e52-aa70-550b7a211d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.519669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-79aa85a7-2c11-4e52-aa70-550b7a211d94 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:08:50.521317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-79aa85a7-2c11-4e52-aa70-550b7a211d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:50.546568 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1005/2033] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:08:50.549703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29311358-bc7b-4a52-b62e-e3570b70ba1a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:50.563201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29311358-bc7b-4a52-b62e-e3570b70ba1a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:50.575233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:50.576089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.576363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.576626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.577273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:08:50.577570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:50.579317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:50.592660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1029/2034] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:08:50.597232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa85a7-2c11-4e52-aa70-550b7a211d94 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:50.606605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa85a7-2c11-4e52-aa70-550b7a211d94 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:50.612489 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:50.613246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.613685 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.614840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.615725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:50.618671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:50.658455 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1006/2035] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:08:50 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:08:50.661825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:50.675067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39ecf8-386e-4b2f-813a-7c53067cb110 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:50.777090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:50.777899 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.778004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.778405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.779425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:50.781691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:50.792699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:08:50.815835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1030/2036] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:08:50 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:08:50.816242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:50.829624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b58dc-4871-44e4-9e7c-37490ef91275 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:50.840147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:50.841472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.842084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.842787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.843561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:50.845628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:50.857503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1007/2037] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:08:50.861600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:50.882650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee3d93-c7ee-4b2a-976f-58f61022d139 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:50.893911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd381a76-e4e6-4bcd-89b1-2aeed8d0d04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:50.895238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd381a76-e4e6-4bcd-89b1-2aeed8d0d04a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.895761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd381a76-e4e6-4bcd-89b1-2aeed8d0d04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.896255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd381a76-e4e6-4bcd-89b1-2aeed8d0d04a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.898353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd381a76-e4e6-4bcd-89b1-2aeed8d0d04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:50.901144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd381a76-e4e6-4bcd-89b1-2aeed8d0d04a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:50.913906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1031/2038] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:08:50.918217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:50.928300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc9c05-2ebd-40a0-9337-a60be473001a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:50.934224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:50.935265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:50.935477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:50.936029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] PATH_INFO: `/v3/projects/d0ab1787783a484b9faeac87feee317f/users/61aa9bba695745d28721e2a1a1cc963b/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:50.953385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ab1787783a484b9faeac87feee317f, user_id=61aa9bba695745d28721e2a1a1cc963b, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:50.956494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:50.995324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1008/2039] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:08:50 2022] PUT /identity/v3/projects/d0ab1787783a484b9faeac87feee317f/users/61aa9bba695745d28721e2a1a1cc963b/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:08:51.051527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:51.051527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:51.051527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:51.086398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] MFA Rules not processed for user `61aa9bba695745d28721e2a1a1cc963b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:08:51.087694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:51.166167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1032/2040] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:08:51.195063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:08:51.198161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:08:51.275254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c913be-0098-4887-8396-d3f0a758a425 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:08:51.279415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54ee2-e7df-4c6e-ac9d-63f336287f5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:08:51.280024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54ee2-e7df-4c6e-ac9d-63f336287f5a service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:08:51.280295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54ee2-e7df-4c6e-ac9d-63f336287f5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:08:51.280565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54ee2-e7df-4c6e-ac9d-63f336287f5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:08:51.293328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54ee2-e7df-4c6e-ac9d-63f336287f5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:08:51.294782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54ee2-e7df-4c6e-ac9d-63f336287f5a service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:08:51.305042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1009/2041] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:08:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:01.161751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:01.300816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:07.671718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:07.673322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:07.754314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c072b1-2bde-4ba1-a546-a6423c463033 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:07.760133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:07.760769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:07.761006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:07.761274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:07.762966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:07.774942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:07.776597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:07.787455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1033/2042] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:09:07.934880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e65c6-e914-4281-b3a4-104001698f20 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:07.935434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e65c6-e914-4281-b3a4-104001698f20 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:07.935434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e65c6-e914-4281-b3a4-104001698f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:07.957439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0d9e65c6-e914-4281-b3a4-104001698f20 None None] MFA Rules not processed for user `1e37d9ed3751483ebe4141706f46e43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:07.958806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0d9e65c6-e914-4281-b3a4-104001698f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:08.025656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1010/2043] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:08.327590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:08.328957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:08.382196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30250d-61a3-42e5-b74e-073fbb64a4ec service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:08.386233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e2a51-b9dc-4cef-8a9c-c87d3d488f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:08.386816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e2a51-b9dc-4cef-8a9c-c87d3d488f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:08.387088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e2a51-b9dc-4cef-8a9c-c87d3d488f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:08.387383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e2a51-b9dc-4cef-8a9c-c87d3d488f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:08.389041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a84e2a51-b9dc-4cef-8a9c-c87d3d488f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:08.397987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e2a51-b9dc-4cef-8a9c-c87d3d488f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:08.399965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e2a51-b9dc-4cef-8a9c-c87d3d488f0d service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:08.407852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1034/2044] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:09.886139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e65c6-e914-4281-b3a4-104001698f20 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:09.992703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e65c6-e914-4281-b3a4-104001698f20 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:10.000744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:10.001673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.002224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.002402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] PATH_INFO: `/v3/users/61aa9bba695745d28721e2a1a1cc963b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.010730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61aa9bba695745d28721e2a1a1cc963b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:10.016737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:10.083969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61aa9bba695745d28721e2a1a1cc963b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.084811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61aa9bba695745d28721e2a1a1cc963b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.094117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61aa9bba695745d28721e2a1a1cc963b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.104356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61aa9bba695745d28721e2a1a1cc963b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.104991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61aa9bba695745d28721e2a1a1cc963b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.122757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1011/2045] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:09 2022] DELETE /identity/v3/users/61aa9bba695745d28721e2a1a1cc963b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:10.127371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.128422 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.128703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.147628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:10.148846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:10.233055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1035/2046] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:10.241161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:10.242549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:10.271074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:10.303269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd792f-1e33-497e-be4a-8d004e760599 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:10.307522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7c445-5056-4f9e-881b-ae1376970735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:10.308144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7c445-5056-4f9e-881b-ae1376970735 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.308423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7c445-5056-4f9e-881b-ae1376970735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.308694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7c445-5056-4f9e-881b-ae1376970735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.318190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7c445-5056-4f9e-881b-ae1376970735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:10.319602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7c445-5056-4f9e-881b-ae1376970735 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:10.329857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1012/2047] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:10.612182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:10.613614 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:10.626445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e04c45-3130-46c2-ae93-8db181660f8d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:10.631772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:10.632489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.632678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.632949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] PATH_INFO: `/v3/projects/d0ab1787783a484b9faeac87feee317f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.634649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ab1787783a484b9faeac87feee317f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:10.636645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:10.662418 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:10.686159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ab1787783a484b9faeac87feee317f', 'request_id': 'req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.691953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ab1787783a484b9faeac87feee317f', 'request_id': 'req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.698787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ab1787783a484b9faeac87feee317f', 'request_id': 'req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.712727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] The token cache is being invalidate because project d0ab1787783a484b9faeac87feee317f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:10.712988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.713656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-baee1342-00ea-41fa-bf3c-cb3e2361cbc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:10.715424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1036/2048] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:10 2022] DELETE /identity/v3/projects/d0ab1787783a484b9faeac87feee317f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:10.720687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c0015-3443-4fcb-8fef-abc27fbd967d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.720995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c0015-3443-4fcb-8fef-abc27fbd967d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.721271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c0015-3443-4fcb-8fef-abc27fbd967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.745599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a93c0015-3443-4fcb-8fef-abc27fbd967d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:10.746797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a93c0015-3443-4fcb-8fef-abc27fbd967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:10.807615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1013/2049] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:10.815351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680dd7-6710-4df3-a798-641776359dfd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.815636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680dd7-6710-4df3-a798-641776359dfd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.815911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680dd7-6710-4df3-a798-641776359dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.838265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7a680dd7-6710-4df3-a798-641776359dfd None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:10.840050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7a680dd7-6710-4df3-a798-641776359dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:10.848020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1037/2050] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:10.852202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c0015-3443-4fcb-8fef-abc27fbd967d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:10.861805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c0015-3443-4fcb-8fef-abc27fbd967d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:10.866374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:10.866997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.867317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.867588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.869075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:09:10.869379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:10.871010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:10.883250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1014/2051] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:09:10.888165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a680dd7-6710-4df3-a798-641776359dfd None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:10.897845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a680dd7-6710-4df3-a798-641776359dfd None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:10.902438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:10.903027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:10.903330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:10.903597 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:10.904225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:10.906108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:10.933576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1038/2052] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:10 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:09:10.937952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:10.948625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454ee0b-b913-4450-81ca-1b083f2e7ce0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:11.011884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:11.012510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:11.012776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:11.013050 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:11.013692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:11.015647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:11.041624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1015/2053] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:10 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:09:11.045912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:11.057872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ad11-ac9e-474d-a912-13765c256d83 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:11.066539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:11.067306 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:11.067712 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:11.068085 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:11.068840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:11.070577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:11.090770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1039/2054] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:11.094903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:11.114017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a9497-63be-4eee-a2ba-a478d45620ef admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:11.120607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751a13d-bb75-408f-a269-990095df8b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:11.121001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751a13d-bb75-408f-a269-990095df8b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:11.121390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751a13d-bb75-408f-a269-990095df8b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:11.121740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751a13d-bb75-408f-a269-990095df8b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:11.122495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751a13d-bb75-408f-a269-990095df8b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:11.125791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751a13d-bb75-408f-a269-990095df8b95 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:11.139243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1016/2055] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:11.142921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:11.154719 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410905c2-b9ea-46e0-8482-93bf5af4bc9d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:11.160397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:11.161058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:11.161336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:11.161611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] PATH_INFO: `/v3/projects/3326642f5c9e4103aa3c72f47b5d23b5/users/455a8022fa3543b2a0ce0cdfc1512f73/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:11.179187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3326642f5c9e4103aa3c72f47b5d23b5, user_id=455a8022fa3543b2a0ce0cdfc1512f73, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:11.182586 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:11.195554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1040/2056] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:11 2022] PUT /identity/v3/projects/3326642f5c9e4103aa3c72f47b5d23b5/users/455a8022fa3543b2a0ce0cdfc1512f73/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:11.201444 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d140259-6a8c-41b8-a124-45f814fd5990 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:11.201817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d140259-6a8c-41b8-a124-45f814fd5990 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:11.202119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d140259-6a8c-41b8-a124-45f814fd5990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:11.222066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0d140259-6a8c-41b8-a124-45f814fd5990 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:11.223330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0d140259-6a8c-41b8-a124-45f814fd5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:11.260386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:11.301819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1017/2057] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:11.312218 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:11.315156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:11.448072 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d61f-f158-4071-ab7e-ba588f99d7c8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:11.452702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:11.453725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:11.454180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:11.454723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:11.465854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:11.467500 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:11.479994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1041/2058] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:17.295893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928f032-f5c1-4d7e-9f4a-0d8e4c43e5bb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:17.296785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928f032-f5c1-4d7e-9f4a-0d8e4c43e5bb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:17.297252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928f032-f5c1-4d7e-9f4a-0d8e4c43e5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:17.335565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b928f032-f5c1-4d7e-9f4a-0d8e4c43e5bb None None] MFA Rules not processed for user `455a8022fa3543b2a0ce0cdfc1512f73`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:17.337469 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b928f032-f5c1-4d7e-9f4a-0d8e4c43e5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:17.412006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1018/2059] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:17.416667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:17.426463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f563c-53a3-4c0e-b820-37472bf73a77 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:17.431683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:17.432301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:17.432576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:17.432845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:17.433478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:17.435413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:17.475198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1042/2060] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:17 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:09:17.480473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928f032-f5c1-4d7e-9f4a-0d8e4c43e5bb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:17.493112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928f032-f5c1-4d7e-9f4a-0d8e4c43e5bb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:17.564957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:17.565580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:17.565896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:17.566158 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:17.566797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:17.568785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:17.598089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1019/2061] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:17 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:17.599449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:17.611328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587e7a5-341f-4c1d-a042-59387f4114bc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:17.620731 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:17.621357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:17.621870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:17.622326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:17.623121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:17.624696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:17.643025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1043/2062] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:09:17.646006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:17.662051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423cc30-09d1-4c9d-8ee3-5da7720dd4f6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:17.691814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:17.692580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:17.692889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:17.693160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] PATH_INFO: `/v3/projects/408f1ef15360431c92948d09ad9b7336/users/389ac9a127eb471884975a97fb5ee026/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:17.718862 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=408f1ef15360431c92948d09ad9b7336, user_id=389ac9a127eb471884975a97fb5ee026, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:17.722141 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:17.737730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1020/2063] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:17 2022] PUT /identity/v3/projects/408f1ef15360431c92948d09ad9b7336/users/389ac9a127eb471884975a97fb5ee026/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:17.741364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:17.757187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c238c93-64a9-429d-b0de-537eeed2017b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:17.773005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:17.845400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:17.845977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:17.846356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:17.846830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:17.847703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:17.849610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:17.863837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1044/2064] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:09:17.868797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:17.890466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d834f-1b68-4cd9-88a9-cfcffabaca4b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:17.902553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418798e-d9f5-4398-8c06-b5dea5b7aedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:17.903398 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418798e-d9f5-4398-8c06-b5dea5b7aedb admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:17.903798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418798e-d9f5-4398-8c06-b5dea5b7aedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:17.904188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418798e-d9f5-4398-8c06-b5dea5b7aedb admin admin] PATH_INFO: `/v3/projects/408f1ef15360431c92948d09ad9b7336/users/389ac9a127eb471884975a97fb5ee026/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:17.906484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418798e-d9f5-4398-8c06-b5dea5b7aedb admin admin] RBAC: Authorizing `identity:create_grant(project_id=408f1ef15360431c92948d09ad9b7336, user_id=389ac9a127eb471884975a97fb5ee026, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:17.910464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418798e-d9f5-4398-8c06-b5dea5b7aedb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:17.925246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1021/2065] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:17 2022] PUT /identity/v3/projects/408f1ef15360431c92948d09ad9b7336/users/389ac9a127eb471884975a97fb5ee026/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:20.117405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:20.122424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:20.269875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb5afa-2f64-4875-8c7a-2b762609ae8a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:20.372651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:20.379868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:20.380932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:20.381400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:20.384130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:20.487027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:20.489186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:20.503489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1045/2066] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:09:23.668562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:23.669726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:23.670388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:23.734416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] MFA Rules not processed for user `389ac9a127eb471884975a97fb5ee026`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:23.736731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:23.741793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:23.911018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1022/2067] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:23.924754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:23.926181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:23.958364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:24.077565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20d92-7f0e-4ecb-ac8e-1d36ed7d7f34 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:24.102175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8e059-1eef-40b9-9e4b-bbc451454aa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:24.102908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e059-1eef-40b9-9e4b-bbc451454aa7 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:24.103286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e059-1eef-40b9-9e4b-bbc451454aa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:24.103653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e059-1eef-40b9-9e4b-bbc451454aa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:24.213240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8e059-1eef-40b9-9e4b-bbc451454aa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:24.215928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8e059-1eef-40b9-9e4b-bbc451454aa7 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:24.230943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1046/2068] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:25.725287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:25.727918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:25.749686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef197060-1922-4c4a-88aa-d5ae91d24dcc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:25.757451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:25.758988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:25.760125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:25.760848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] PATH_INFO: `/v3/users/c987a5a52fc641f78054ac0218edcd13` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:25.762541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c987a5a52fc641f78054ac0218edcd13)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:25.765604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:25.845799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c987a5a52fc641f78054ac0218edcd13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:25.847474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c987a5a52fc641f78054ac0218edcd13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:25.858377 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c987a5a52fc641f78054ac0218edcd13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:25.872908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c987a5a52fc641f78054ac0218edcd13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:25.873924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c987a5a52fc641f78054ac0218edcd13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:25.899700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1023/2069] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:25 2022] DELETE /identity/v3/users/c987a5a52fc641f78054ac0218edcd13 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:26.251723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb771fd7-26dc-4aa1-93a3-14da7c611ac6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:26.252559 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb771fd7-26dc-4aa1-93a3-14da7c611ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:26.252994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb771fd7-26dc-4aa1-93a3-14da7c611ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:26.267969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:26.270981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:26.330957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-eb771fd7-26dc-4aa1-93a3-14da7c611ac6 None None] MFA Rules not processed for user `455a8022fa3543b2a0ce0cdfc1512f73`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:26.334163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eb771fd7-26dc-4aa1-93a3-14da7c611ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:26.409143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43709b-fbdc-4e11-93ee-6b04651b0a7b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:26.421465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:26.422400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:26.422938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:26.423430 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] PATH_INFO: `/v3/projects/0cb1aa55658342fcaf2f493f51e9faeb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:26.428578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1047/2070] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:26.428801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb1aa55658342fcaf2f493f51e9faeb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:26.429090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:26.472951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:26.526936 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb1aa55658342fcaf2f493f51e9faeb', 'request_id': 'req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:26.533729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb1aa55658342fcaf2f493f51e9faeb', 'request_id': 'req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:26.545684 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb1aa55658342fcaf2f493f51e9faeb', 'request_id': 'req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:26.564660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] The token cache is being invalidate because project 0cb1aa55658342fcaf2f493f51e9faeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:26.564957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:26.565809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f6964c9-fc72-4ab2-9ca4-9555aa98b218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:26.568011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1024/2071] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:26 2022] DELETE /identity/v3/projects/0cb1aa55658342fcaf2f493f51e9faeb => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:26.573116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e0f05-adba-4323-a64e-7f1c574b67c5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:26.573533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e0f05-adba-4323-a64e-7f1c574b67c5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:26.574156 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e0f05-adba-4323-a64e-7f1c574b67c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:26.608476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-182e0f05-adba-4323-a64e-7f1c574b67c5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:26.609724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-182e0f05-adba-4323-a64e-7f1c574b67c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:26.703923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1048/2072] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:26.704847 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0826c-0087-4d57-b197-854688a60230 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:26.705126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0826c-0087-4d57-b197-854688a60230 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:26.705391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0826c-0087-4d57-b197-854688a60230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:26.728433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e5e0826c-0087-4d57-b197-854688a60230 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:26.740232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e5e0826c-0087-4d57-b197-854688a60230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:26.757163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1025/2073] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:26.757707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e0f05-adba-4323-a64e-7f1c574b67c5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:26.763512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:26.772237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e0f05-adba-4323-a64e-7f1c574b67c5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:26.787282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:26.788234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:26.788720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:26.789221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:26.790290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:09:26.790788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:26.793405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:26.813058 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1049/2074] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:09:26.814086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0826c-0087-4d57-b197-854688a60230 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:26.831985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0826c-0087-4d57-b197-854688a60230 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:26.837095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:26.837896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:26.838325 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:26.838752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:26.839416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:26.841761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:26.918158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:26.927619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1026/2075] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:26 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:26.946152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7a6fd-6e6d-411e-86ea-a8f8c1aba87f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.092060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.092791 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.093347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.093636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.095464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.099004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.145956 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1050/2076] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:26 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:27.151856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.168197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c45611-d6a5-406f-8395-34a928ff99dd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.173244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.173865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.174167 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.174440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.175100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.177149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.193576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1027/2077] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:27.194175 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.209513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69967a-089c-4b76-859c-a5d899a8a51f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.217227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4345f10-5f56-4bad-bd26-de4e1016038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.217988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4345f10-5f56-4bad-bd26-de4e1016038f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.218370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4345f10-5f56-4bad-bd26-de4e1016038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.218774 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4345f10-5f56-4bad-bd26-de4e1016038f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.219531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4345f10-5f56-4bad-bd26-de4e1016038f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.231231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4345f10-5f56-4bad-bd26-de4e1016038f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.236237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1051/2078] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:09:27.240122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.263105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043025cc-a2d0-49bb-864f-ad27b2c7addf admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.269288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.270121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.270378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.270739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] PATH_INFO: `/v3/projects/8dd08d0c6cd0481c8f4bea12ea703198/users/d0ae3e7bb7bd49849def5b8baded25f4/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.287682 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd08d0c6cd0481c8f4bea12ea703198, user_id=d0ae3e7bb7bd49849def5b8baded25f4, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.290486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.310878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1028/2079] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:27 2022] PUT /identity/v3/projects/8dd08d0c6cd0481c8f4bea12ea703198/users/d0ae3e7bb7bd49849def5b8baded25f4/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:27.322995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c850ce2-1a79-48be-b8ba-008f617acae0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.323549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c850ce2-1a79-48be-b8ba-008f617acae0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.323549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c850ce2-1a79-48be-b8ba-008f617acae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.374840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2c850ce2-1a79-48be-b8ba-008f617acae0 None None] MFA Rules not processed for user `d0ae3e7bb7bd49849def5b8baded25f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:27.376257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2c850ce2-1a79-48be-b8ba-008f617acae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:27.449138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1052/2080] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:27.454020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.536561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74caba84-1c8b-4c8b-b877-fd2f8d28730d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.541199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.541940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.542124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.542394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.543029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.544958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.576219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1029/2081] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:27 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:27.580715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c850ce2-1a79-48be-b8ba-008f617acae0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.592674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c850ce2-1a79-48be-b8ba-008f617acae0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.647893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:27.655660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.656478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.656767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.657260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.657954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.659984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.685797 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1053/2082] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:27 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:27.687287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.703549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58e8c-d9d9-4f5a-8c33-037d0e5deda8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.708278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.708890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.709166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.709436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.710096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.712317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.723638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1030/2083] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:27.728038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.739605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc62d8-e304-47e9-bb1b-31c20e6b6764 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.747270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.747982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.748259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.748531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] PATH_INFO: `/v3/projects/8bd6a322c5f84626b49c0607d1601116/users/6e47224208224ba8a8d8db71860da35d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.764158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd6a322c5f84626b49c0607d1601116, user_id=6e47224208224ba8a8d8db71860da35d, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.766987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.781645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1054/2084] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:27 2022] PUT /identity/v3/projects/8bd6a322c5f84626b49c0607d1601116/users/6e47224208224ba8a8d8db71860da35d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:27.785921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.798090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593044b8-4511-4546-a318-679cfe3f5786 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.836800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:27.894653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9526c0e-28b4-4166-9884-6441ab697e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.895285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9526c0e-28b4-4166-9884-6441ab697e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.895560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9526c0e-28b4-4166-9884-6441ab697e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.895828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9526c0e-28b4-4166-9884-6441ab697e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.896465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9526c0e-28b4-4166-9884-6441ab697e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.898091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9526c0e-28b4-4166-9884-6441ab697e79 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.915300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1031/2085] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:27.916540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:27.927119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6759b-61ee-499c-a258-eced0294f5f0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:27.938415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:27.939064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.939374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.939646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] PATH_INFO: `/v3/projects/8bd6a322c5f84626b49c0607d1601116/users/6e47224208224ba8a8d8db71860da35d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.941689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd6a322c5f84626b49c0607d1601116, user_id=6e47224208224ba8a8d8db71860da35d, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:27.944374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:27.956424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1055/2086] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:27 2022] PUT /identity/v3/projects/8bd6a322c5f84626b49c0607d1601116/users/6e47224208224ba8a8d8db71860da35d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:27.967252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e878-9dd9-4821-9b09-bb9f7d5f6229 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:27.967569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e878-9dd9-4821-9b09-bb9f7d5f6229 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:27.967801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e878-9dd9-4821-9b09-bb9f7d5f6229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:27.998940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5105e878-9dd9-4821-9b09-bb9f7d5f6229 None None] MFA Rules not processed for user `6e47224208224ba8a8d8db71860da35d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:28.000302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5105e878-9dd9-4821-9b09-bb9f7d5f6229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:28.077104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1032/2087] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:28.089400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:28.091170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:28.176481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8fd71-6646-4c07-8c61-917510d24764 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:28.180274 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac7dfd-d00f-44f4-a409-a3e75f1c3088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:28.180923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7dfd-d00f-44f4-a409-a3e75f1c3088 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:28.181178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7dfd-d00f-44f4-a409-a3e75f1c3088 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:28.181480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7dfd-d00f-44f4-a409-a3e75f1c3088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:28.206270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac7dfd-d00f-44f4-a409-a3e75f1c3088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:28.207804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac7dfd-d00f-44f4-a409-a3e75f1c3088 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:28.223691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1056/2088] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:28.256297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e878-9dd9-4821-9b09-bb9f7d5f6229 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:28.332117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e878-9dd9-4821-9b09-bb9f7d5f6229 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:28.337257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:28.337960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:28.338335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:28.338715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] PATH_INFO: `/v3/users/d0ae3e7bb7bd49849def5b8baded25f4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:28.339996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ae3e7bb7bd49849def5b8baded25f4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:28.342028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:28.405226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:28.423992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ae3e7bb7bd49849def5b8baded25f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:28.424705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ae3e7bb7bd49849def5b8baded25f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:28.434159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ae3e7bb7bd49849def5b8baded25f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:28.441949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ae3e7bb7bd49849def5b8baded25f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:28.442764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ae3e7bb7bd49849def5b8baded25f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:28.471463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1033/2089] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:28 2022] DELETE /identity/v3/users/d0ae3e7bb7bd49849def5b8baded25f4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:28.472756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:28.473242 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:28.473341 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:28.500443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:28.501694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:28.571591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1057/2090] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:28.585121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:28.586585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:28.654347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2031308-4703-4e1d-9503-7f6417727e36 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:28.658028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:28.658650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:28.658911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:28.659177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:28.669351 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:28.670808 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:28.682835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1034/2091] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:29.252042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:29.253844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:29.271088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dea81-4600-4e5b-b4a7-f7c8cad5f446 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:29.277504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:29.278144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:29.278419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:29.278680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] PATH_INFO: `/v3/projects/8dd08d0c6cd0481c8f4bea12ea703198` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:29.281232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd08d0c6cd0481c8f4bea12ea703198)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:29.283314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:29.354740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:29.357381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd08d0c6cd0481c8f4bea12ea703198', 'request_id': 'req-8ed14a42-b700-406c-b1a0-4fb06cf59b48'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.364589 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd08d0c6cd0481c8f4bea12ea703198', 'request_id': 'req-8ed14a42-b700-406c-b1a0-4fb06cf59b48'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.373182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd08d0c6cd0481c8f4bea12ea703198', 'request_id': 'req-8ed14a42-b700-406c-b1a0-4fb06cf59b48'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.394571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] The token cache is being invalidate because project 8dd08d0c6cd0481c8f4bea12ea703198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:29.394987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.395780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.400885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1058/2092] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:29 2022] DELETE /identity/v3/projects/8dd08d0c6cd0481c8f4bea12ea703198 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:29.402116 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:29.414755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780e534-46bc-43c3-9e7d-4c953951cbba service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:29.488750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:29.489421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:29.489723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:29.490013 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] PATH_INFO: `/v3/users/6e47224208224ba8a8d8db71860da35d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:29.491231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e47224208224ba8a8d8db71860da35d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:29.493602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:29.557236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e47224208224ba8a8d8db71860da35d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.557996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e47224208224ba8a8d8db71860da35d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.565676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e47224208224ba8a8d8db71860da35d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.568264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:29.574888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e47224208224ba8a8d8db71860da35d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.577295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4f0d25c9-cb48-4c84-8129-19b98b17b1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e47224208224ba8a8d8db71860da35d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:29.600698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1035/2093] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:29 2022] DELETE /identity/v3/users/6e47224208224ba8a8d8db71860da35d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:29.894281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:29.895951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:29.913133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed14a42-b700-406c-b1a0-4fb06cf59b48 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:29.991193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:29.991966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:29.992907 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:29.993183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] PATH_INFO: `/v3/projects/8bd6a322c5f84626b49c0607d1601116` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:29.994420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd6a322c5f84626b49c0607d1601116)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:29.996999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.059398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd6a322c5f84626b49c0607d1601116', 'request_id': 'req-d9493c47-c922-415e-aad2-61a20befce36'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:30.066007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd6a322c5f84626b49c0607d1601116', 'request_id': 'req-d9493c47-c922-415e-aad2-61a20befce36'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:30.073737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd6a322c5f84626b49c0607d1601116', 'request_id': 'req-d9493c47-c922-415e-aad2-61a20befce36'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:30.091858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] The token cache is being invalidate because project 8bd6a322c5f84626b49c0607d1601116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:30.092243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:30.098581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9493c47-c922-415e-aad2-61a20befce36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:30.098581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1059/2094] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:29 2022] DELETE /identity/v3/projects/8bd6a322c5f84626b49c0607d1601116 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:30.104127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fd31e-d7d3-4b9c-8ec4-a16eec564cde None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.104410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fd31e-d7d3-4b9c-8ec4-a16eec564cde None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.104680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fd31e-d7d3-4b9c-8ec4-a16eec564cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.125022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-588fd31e-d7d3-4b9c-8ec4-a16eec564cde None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:30.126310 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-588fd31e-d7d3-4b9c-8ec4-a16eec564cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:30.229481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1036/2095] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:30.237057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b2946-e2f3-438c-8278-55166de1e159 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.237339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b2946-e2f3-438c-8278-55166de1e159 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.237615 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b2946-e2f3-438c-8278-55166de1e159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.268359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c35b2946-e2f3-438c-8278-55166de1e159 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:30.272499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c35b2946-e2f3-438c-8278-55166de1e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:30.282725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1060/2096] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:30.286817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fd31e-d7d3-4b9c-8ec4-a16eec564cde None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.297498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fd31e-d7d3-4b9c-8ec4-a16eec564cde None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:30.302374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:30.302977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.303285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.303559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.304203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:09:30.304504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:30.306209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.325673 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1037/2097] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:09:30.327044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b2946-e2f3-438c-8278-55166de1e159 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.337447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b2946-e2f3-438c-8278-55166de1e159 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:30.342515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:30.343223 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.343574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.343919 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.344617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:30.346594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.389588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1061/2098] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:30 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:30.390756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.401285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d07f1-02e1-4f5b-b4b6-8ceca76c3170 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:30.472739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:30.474087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.474407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.474683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.475427 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:30.478200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.505503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1038/2099] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:30 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:30.509640 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.521247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c94f7-9ea9-4365-b663-6b3900f5e3d7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:30.526506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:30.527181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.527477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.527749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.528396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:30.530041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.541673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1062/2100] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:09:30.545626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.556871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe28a7-2002-4385-98c7-2964a41136ca admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:30.561735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58b7b4-1b31-4c83-b1d1-39d263793f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:30.562346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58b7b4-1b31-4c83-b1d1-39d263793f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.562625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58b7b4-1b31-4c83-b1d1-39d263793f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.562897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58b7b4-1b31-4c83-b1d1-39d263793f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.563567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58b7b4-1b31-4c83-b1d1-39d263793f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:30.565171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58b7b4-1b31-4c83-b1d1-39d263793f04 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.580947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1039/2101] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:30.585319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.598282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a59ab3-629d-43db-89c0-ad2ed5c980b6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:30.605187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:30.605849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.606256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.606536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] PATH_INFO: `/v3/projects/fcdee1ea4fa84ee996601d28d6083626/users/2cbd4239b17e418aa476938b4f8e6d2e/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.624611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdee1ea4fa84ee996601d28d6083626, user_id=2cbd4239b17e418aa476938b4f8e6d2e, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:30.627439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.647997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1063/2102] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:30 2022] PUT /identity/v3/projects/fcdee1ea4fa84ee996601d28d6083626/users/2cbd4239b17e418aa476938b4f8e6d2e/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:30.659422 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d53a-0fcf-494a-a59d-ceb8b02351f8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.659757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d53a-0fcf-494a-a59d-ceb8b02351f8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.659992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d53a-0fcf-494a-a59d-ceb8b02351f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.693317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9330d53a-0fcf-494a-a59d-ceb8b02351f8 None None] MFA Rules not processed for user `2cbd4239b17e418aa476938b4f8e6d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:30.694796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9330d53a-0fcf-494a-a59d-ceb8b02351f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:30.730715 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:30.780461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1040/2103] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:30.784926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.789805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:30.872991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d571-024f-4136-bba7-6f656cd7fad5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:30.879595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:30.880222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:30.880511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:30.880756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:30.881401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:30.883969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:30.927073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1064/2104] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:30 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:09:30.928332 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d53a-0fcf-494a-a59d-ceb8b02351f8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:30.944021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d53a-0fcf-494a-a59d-ceb8b02351f8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.018718 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.019425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.019705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.019977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.020611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.022556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.045408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1041/2105] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:30 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:31.056183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:31.068354 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9683a4e-d92e-4427-ac10-a80128d79c7b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.074827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.076186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.076186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.076372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.076937 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.078519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.089910 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1065/2106] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:31.094564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:31.107909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9df39-6f7c-4706-8c9d-5d4143723abc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.112603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.113263 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.113569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.113861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] PATH_INFO: `/v3/projects/8798d6c22bf64d729793a31c73dedca1/users/00f1a26a9d6241d09f02e2a37958f5af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.129833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8798d6c22bf64d729793a31c73dedca1, user_id=00f1a26a9d6241d09f02e2a37958f5af, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.132560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.145764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1042/2107] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:31 2022] PUT /identity/v3/projects/8798d6c22bf64d729793a31c73dedca1/users/00f1a26a9d6241d09f02e2a37958f5af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:31.149883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:31.160129 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a90b18-3b53-41cb-ba4e-ac269605884b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.217220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13667-bc8c-466e-915e-c579546fa46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.217837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13667-bc8c-466e-915e-c579546fa46d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.218114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13667-bc8c-466e-915e-c579546fa46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.218420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13667-bc8c-466e-915e-c579546fa46d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.219228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13667-bc8c-466e-915e-c579546fa46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.220917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13667-bc8c-466e-915e-c579546fa46d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.232111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1066/2108] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:31.236127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:31.246608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162975-3c2a-4215-b955-9b8136952ae0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.251458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.252107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.252386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.252662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] PATH_INFO: `/v3/projects/8798d6c22bf64d729793a31c73dedca1/users/00f1a26a9d6241d09f02e2a37958f5af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.254556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8798d6c22bf64d729793a31c73dedca1, user_id=00f1a26a9d6241d09f02e2a37958f5af, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.257405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.268447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1043/2109] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:31 2022] PUT /identity/v3/projects/8798d6c22bf64d729793a31c73dedca1/users/00f1a26a9d6241d09f02e2a37958f5af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:31.279399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c625dc9-5268-41b0-b1b4-7fa09f677f5f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.279688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c625dc9-5268-41b0-b1b4-7fa09f677f5f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.279963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c625dc9-5268-41b0-b1b4-7fa09f677f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.309620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0c625dc9-5268-41b0-b1b4-7fa09f677f5f None None] MFA Rules not processed for user `00f1a26a9d6241d09f02e2a37958f5af`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:31.310827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0c625dc9-5268-41b0-b1b4-7fa09f677f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:31.374184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:31.380330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1067/2110] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:31.391022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:31.393356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:31.452868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcb12a-70f8-4c5a-b1f4-12c736a5cb3f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.456511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f103a-3568-4ec8-9a82-a7970c323244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.457107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f103a-3568-4ec8-9a82-a7970c323244 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.457380 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f103a-3568-4ec8-9a82-a7970c323244 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.457654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f103a-3568-4ec8-9a82-a7970c323244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.467266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f103a-3568-4ec8-9a82-a7970c323244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.468770 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f103a-3568-4ec8-9a82-a7970c323244 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.479056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1044/2111] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:31.621760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c625dc9-5268-41b0-b1b4-7fa09f677f5f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:31.688117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c625dc9-5268-41b0-b1b4-7fa09f677f5f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.692601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.693252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.693526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.693795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] PATH_INFO: `/v3/users/2cbd4239b17e418aa476938b4f8e6d2e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.694906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cbd4239b17e418aa476938b4f8e6d2e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.697468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.746715 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cbd4239b17e418aa476938b4f8e6d2e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:31.747312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cbd4239b17e418aa476938b4f8e6d2e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:31.754225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cbd4239b17e418aa476938b4f8e6d2e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:31.754888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cbd4239b17e418aa476938b4f8e6d2e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:31.767063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cbd4239b17e418aa476938b4f8e6d2e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:31.779865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1068/2112] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:31 2022] DELETE /identity/v3/users/2cbd4239b17e418aa476938b4f8e6d2e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:31.784607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.784962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.785334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.804979 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:31.806211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:31.829289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:31.873721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1045/2113] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:31.883889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:31.885520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:31.951062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c83b6d-f6e6-4465-bb80-e18e553aa52b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:31.956352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:31.957043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:31.957464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:31.957879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:31.968066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:31.969523 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:31.979820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1069/2114] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:32.266814 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:32.268287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:32.282417 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6faa7a-1586-4a97-9f53-660ae1c11a6e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:32.287137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:32.287798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:32.288078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:32.288350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] PATH_INFO: `/v3/projects/fcdee1ea4fa84ee996601d28d6083626` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:32.289466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcdee1ea4fa84ee996601d28d6083626)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:32.292507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:32.345775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcdee1ea4fa84ee996601d28d6083626', 'request_id': 'req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.351696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcdee1ea4fa84ee996601d28d6083626', 'request_id': 'req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.359676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcdee1ea4fa84ee996601d28d6083626', 'request_id': 'req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.372544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] The token cache is being invalidate because project fcdee1ea4fa84ee996601d28d6083626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:32.372841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.373575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.375111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1046/2115] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:32 2022] DELETE /identity/v3/projects/fcdee1ea4fa84ee996601d28d6083626 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:32.379084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:32.389406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af3951-8366-43db-8fac-c2578b0e0266 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:32.405199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:32.449061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:32.449687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:32.449965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:32.450236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] PATH_INFO: `/v3/users/00f1a26a9d6241d09f02e2a37958f5af` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:32.451630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f1a26a9d6241d09f02e2a37958f5af)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:32.453834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:32.498284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f1a26a9d6241d09f02e2a37958f5af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.498976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f1a26a9d6241d09f02e2a37958f5af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.506022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f1a26a9d6241d09f02e2a37958f5af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.506679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f1a26a9d6241d09f02e2a37958f5af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.518827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71ab0bc9-f2cb-4e28-a3d7-269481c608b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f1a26a9d6241d09f02e2a37958f5af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.531110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1070/2116] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:32 2022] DELETE /identity/v3/users/00f1a26a9d6241d09f02e2a37958f5af => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:32.798278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:32.799798 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:32.813851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9dab5-a3d8-40ea-b240-45e2591d89d5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:32.874567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:32.875231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:32.875510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:32.875786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] PATH_INFO: `/v3/projects/8798d6c22bf64d729793a31c73dedca1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:32.876919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8798d6c22bf64d729793a31c73dedca1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:32.878831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:32.930776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8798d6c22bf64d729793a31c73dedca1', 'request_id': 'req-5bd251be-1115-4795-83ed-ed5b69c3a1f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.932734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:32.937704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8798d6c22bf64d729793a31c73dedca1', 'request_id': 'req-5bd251be-1115-4795-83ed-ed5b69c3a1f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.945611 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8798d6c22bf64d729793a31c73dedca1', 'request_id': 'req-5bd251be-1115-4795-83ed-ed5b69c3a1f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.959943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] The token cache is being invalidate because project 8798d6c22bf64d729793a31c73dedca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:32.960368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.961072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5bd251be-1115-4795-83ed-ed5b69c3a1f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:32.962640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1047/2117] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:32 2022] DELETE /identity/v3/projects/8798d6c22bf64d729793a31c73dedca1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:32.967606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc83f98-7b3d-4599-8c2f-253ae2643ac5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:32.968060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc83f98-7b3d-4599-8c2f-253ae2643ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:32.968174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc83f98-7b3d-4599-8c2f-253ae2643ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:32.988333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8fc83f98-7b3d-4599-8c2f-253ae2643ac5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:32.989544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8fc83f98-7b3d-4599-8c2f-253ae2643ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:33.055860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1071/2118] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:33.064701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b3393-27bb-4ba8-809f-ecf0f539f629 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.067627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b3393-27bb-4ba8-809f-ecf0f539f629 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.067627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b3393-27bb-4ba8-809f-ecf0f539f629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.086837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2d9b3393-27bb-4ba8-809f-ecf0f539f629 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:33.088062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d9b3393-27bb-4ba8-809f-ecf0f539f629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:33.096114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1048/2119] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:33.100490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc83f98-7b3d-4599-8c2f-253ae2643ac5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.110815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc83f98-7b3d-4599-8c2f-253ae2643ac5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.115394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.115992 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.116292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.116577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.117238 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:09:33.117567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.119152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.132182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1072/2120] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:09:33.136317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b3393-27bb-4ba8-809f-ecf0f539f629 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.146234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b3393-27bb-4ba8-809f-ecf0f539f629 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.151544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.152696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.152696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.152696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.153362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.155393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.187078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1049/2121] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:33 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:33.188219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.198186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ea535-b9a1-44fe-be8f-00065fd65857 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.222213 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:33.258124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.258726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.259002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.259321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.259939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.261847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.286535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1073/2122] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:33 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:33.287675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.299021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192407-4bdd-4ff1-a35c-7427424424df admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.303721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.304324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.304682 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.304956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.305595 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.307307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.317415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1050/2123] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:09:33.321472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.331587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148991a-d283-45bb-a601-800f8a5b3058 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.336585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa6e24-2649-4b75-97f2-6e0156057156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.337164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa6e24-2649-4b75-97f2-6e0156057156 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.337438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa6e24-2649-4b75-97f2-6e0156057156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.337711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa6e24-2649-4b75-97f2-6e0156057156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.338333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa6e24-2649-4b75-97f2-6e0156057156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.339947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa6e24-2649-4b75-97f2-6e0156057156 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.352776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1074/2124] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:33.357268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.367522 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a9fbf-c96a-4056-b243-b3c403142527 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.372203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.372848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.373126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.373401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] PATH_INFO: `/v3/projects/d8fe88034c234976b356c4d60578575e/users/3e7a18740c56427693e13a1cd9b7a28d/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.388788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8fe88034c234976b356c4d60578575e, user_id=3e7a18740c56427693e13a1cd9b7a28d, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.391471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.417018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1051/2125] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:33 2022] PUT /identity/v3/projects/d8fe88034c234976b356c4d60578575e/users/3e7a18740c56427693e13a1cd9b7a28d/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:33.427283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9662-a8e2-4c28-9deb-0514e3c1a0ba None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.427571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9662-a8e2-4c28-9deb-0514e3c1a0ba None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.427850 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9662-a8e2-4c28-9deb-0514e3c1a0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.457123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-acbd9662-a8e2-4c28-9deb-0514e3c1a0ba None None] MFA Rules not processed for user `3e7a18740c56427693e13a1cd9b7a28d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:33.458390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-acbd9662-a8e2-4c28-9deb-0514e3c1a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:33.521968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1075/2126] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:33.527163 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.589960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796a8d7-3fb9-450a-b5f8-9ed007b6841c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.594825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.595467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.595747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.596045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.596681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.598644 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.629445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1052/2127] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:33 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:09:33.633468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9662-a8e2-4c28-9deb-0514e3c1a0ba None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.643960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9662-a8e2-4c28-9deb-0514e3c1a0ba None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.699726 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.700330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.700864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.701130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.701757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.703681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.724514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1076/2128] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:33 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:33.728873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.739319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0f46c-0ec2-41a1-8fae-9714b3645a85 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.745292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.745885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.746165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.746437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.747106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.748752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.762197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1053/2129] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:33.764174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.774779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcab3c-97d7-4057-baf4-29f6e08cd936 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.779668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.780400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.780679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.780953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] PATH_INFO: `/v3/projects/0bccaecd996f4d209c34a0a494d86ccb/users/6ff4502fa59446a49ab9d335ddf13c12/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.795895 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bccaecd996f4d209c34a0a494d86ccb, user_id=6ff4502fa59446a49ab9d335ddf13c12, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.798646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.809067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:33.814367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1077/2130] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:33 2022] PUT /identity/v3/projects/0bccaecd996f4d209c34a0a494d86ccb/users/6ff4502fa59446a49ab9d335ddf13c12/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:33.818087 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.828818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cd406-3c5a-4af7-9076-1074febc0545 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.876904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:33.888661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc00f5-82ee-4ed1-948a-c47199df075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.889264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc00f5-82ee-4ed1-948a-c47199df075e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.889543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc00f5-82ee-4ed1-948a-c47199df075e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.889817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc00f5-82ee-4ed1-948a-c47199df075e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.890452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc00f5-82ee-4ed1-948a-c47199df075e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.892083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc00f5-82ee-4ed1-948a-c47199df075e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.902274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1054/2131] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:33.906474 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:33.917211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f775ea6-ff0b-4929-b341-0c15a44ef8c4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:33.922097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:33.922764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.923060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.923375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] PATH_INFO: `/v3/projects/0bccaecd996f4d209c34a0a494d86ccb/users/6ff4502fa59446a49ab9d335ddf13c12/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.925406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bccaecd996f4d209c34a0a494d86ccb, user_id=6ff4502fa59446a49ab9d335ddf13c12, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:33.928113 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:33.940042 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1078/2132] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:33 2022] PUT /identity/v3/projects/0bccaecd996f4d209c34a0a494d86ccb/users/6ff4502fa59446a49ab9d335ddf13c12/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:33.951000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b5a9-8d0b-411b-90eb-336df9b4e1d9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:33.951342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b5a9-8d0b-411b-90eb-336df9b4e1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:33.951744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b5a9-8d0b-411b-90eb-336df9b4e1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:33.986170 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6e22b5a9-8d0b-411b-90eb-336df9b4e1d9 None None] MFA Rules not processed for user `6ff4502fa59446a49ab9d335ddf13c12`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:33.987063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6e22b5a9-8d0b-411b-90eb-336df9b4e1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:34.061921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1055/2133] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:34.073233 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:34.075077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:34.138158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a3a66-c996-4651-9f82-b68b29673e5b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:34.142023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eff83a-6371-4db8-9721-218b9ddc0d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:34.142623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eff83a-6371-4db8-9721-218b9ddc0d83 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:34.142898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eff83a-6371-4db8-9721-218b9ddc0d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:34.143179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eff83a-6371-4db8-9721-218b9ddc0d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:34.153243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eff83a-6371-4db8-9721-218b9ddc0d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:34.154661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eff83a-6371-4db8-9721-218b9ddc0d83 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:34.164252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1079/2134] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:34.550371 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22b5a9-8d0b-411b-90eb-336df9b4e1d9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:34.612609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22b5a9-8d0b-411b-90eb-336df9b4e1d9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:34.617619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:34.618242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:34.618515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:34.618779 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] PATH_INFO: `/v3/users/3e7a18740c56427693e13a1cd9b7a28d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:34.619901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7a18740c56427693e13a1cd9b7a28d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:34.621825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:34.670596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7a18740c56427693e13a1cd9b7a28d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:34.671282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7a18740c56427693e13a1cd9b7a28d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:34.678078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7a18740c56427693e13a1cd9b7a28d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:34.685717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7a18740c56427693e13a1cd9b7a28d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:34.686491 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7a18740c56427693e13a1cd9b7a28d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:34.697179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:34.708445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1056/2135] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:34 2022] DELETE /identity/v3/users/3e7a18740c56427693e13a1cd9b7a28d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:34.713330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:34.713642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:34.713912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:34.733641 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:34.734814 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:34.792653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1080/2136] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:34.803094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:34.805257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:34.864414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ca3be-c43f-4ff2-af43-16edcfbdad59 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:34.868164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:34.868771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:34.869047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:34.869318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:34.878767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:34.880234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:34.890016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1057/2137] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:35.178404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:35.180457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:35.192490 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac95f8-4f0a-4488-a827-f28cf23ba551 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:35.197052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:35.197674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:35.197950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:35.198222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] PATH_INFO: `/v3/projects/d8fe88034c234976b356c4d60578575e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:35.199352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8fe88034c234976b356c4d60578575e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:35.201526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:35.251555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8fe88034c234976b356c4d60578575e', 'request_id': 'req-05b53949-8147-480d-b06f-85d46ee68289'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.256936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8fe88034c234976b356c4d60578575e', 'request_id': 'req-05b53949-8147-480d-b06f-85d46ee68289'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.263307 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8fe88034c234976b356c4d60578575e', 'request_id': 'req-05b53949-8147-480d-b06f-85d46ee68289'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.278356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] The token cache is being invalidate because project d8fe88034c234976b356c4d60578575e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:35.278356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.278356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.279484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1081/2138] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:35 2022] DELETE /identity/v3/projects/d8fe88034c234976b356c4d60578575e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:35.283477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:35.293899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2474b8d-9a3d-433b-a319-b71790193db8 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:35.350480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:35.351108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:35.351984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:35.352259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] PATH_INFO: `/v3/users/6ff4502fa59446a49ab9d335ddf13c12` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:35.353404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff4502fa59446a49ab9d335ddf13c12)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:35.355400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:35.400722 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff4502fa59446a49ab9d335ddf13c12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.401625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff4502fa59446a49ab9d335ddf13c12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.407843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:35.409409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:35.410182 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff4502fa59446a49ab9d335ddf13c12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.417766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff4502fa59446a49ab9d335ddf13c12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.418383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff4502fa59446a49ab9d335ddf13c12'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.424977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:35.438307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1058/2139] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:35 2022] DELETE /identity/v3/users/6ff4502fa59446a49ab9d335ddf13c12 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:35.505144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b53949-8147-480d-b06f-85d46ee68289 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:35.582669 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb02d7-7aeb-49d5-8100-43f7a9fc417e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:35.583330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb02d7-7aeb-49d5-8100-43f7a9fc417e service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:35.583609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb02d7-7aeb-49d5-8100-43f7a9fc417e service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:35.583878 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb02d7-7aeb-49d5-8100-43f7a9fc417e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:35.586169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-32fb02d7-7aeb-49d5-8100-43f7a9fc417e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:35.649936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb02d7-7aeb-49d5-8100-43f7a9fc417e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:35.651946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb02d7-7aeb-49d5-8100-43f7a9fc417e service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:35.665345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1082/2140] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:09:35.781138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:35.782570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:35.799447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:35.865303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58268bb7-875e-46cd-9e43-ca88866ae69a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:35.869833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:35.870450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:35.870724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:35.870992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] PATH_INFO: `/v3/projects/0bccaecd996f4d209c34a0a494d86ccb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:35.872142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bccaecd996f4d209c34a0a494d86ccb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:35.874005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:35.929308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bccaecd996f4d209c34a0a494d86ccb', 'request_id': 'req-95f4a812-e7dd-4473-ac2f-d63600b32986'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.934694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bccaecd996f4d209c34a0a494d86ccb', 'request_id': 'req-95f4a812-e7dd-4473-ac2f-d63600b32986'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.944442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bccaecd996f4d209c34a0a494d86ccb', 'request_id': 'req-95f4a812-e7dd-4473-ac2f-d63600b32986'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.957004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] The token cache is being invalidate because project 0bccaecd996f4d209c34a0a494d86ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:35.957289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.957995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95f4a812-e7dd-4473-ac2f-d63600b32986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:35.959520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1059/2141] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:35 2022] DELETE /identity/v3/projects/0bccaecd996f4d209c34a0a494d86ccb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:35.964949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eece3af-b652-4f58-9d9c-fa53201afff5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:35.965251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eece3af-b652-4f58-9d9c-fa53201afff5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:35.965508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eece3af-b652-4f58-9d9c-fa53201afff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:35.984636 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1eece3af-b652-4f58-9d9c-fa53201afff5 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:35.985826 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1eece3af-b652-4f58-9d9c-fa53201afff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:36.033075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:36.048508 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1083/2142] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:36.057104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb1a6-95b6-40e1-aefb-56fbca9aa717 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.057388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb1a6-95b6-40e1-aefb-56fbca9aa717 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.057687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb1a6-95b6-40e1-aefb-56fbca9aa717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.079335 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6d8fb1a6-95b6-40e1-aefb-56fbca9aa717 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:36.080528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6d8fb1a6-95b6-40e1-aefb-56fbca9aa717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:36.088352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1060/2143] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:36.092729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eece3af-b652-4f58-9d9c-fa53201afff5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:36.103856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eece3af-b652-4f58-9d9c-fa53201afff5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:36.108831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:36.109420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.109694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.109961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.110630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:09:36.110928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:36.112554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:36.125588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1084/2144] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:09:36.130047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fb1a6-95b6-40e1-aefb-56fbca9aa717 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:36.139359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fb1a6-95b6-40e1-aefb-56fbca9aa717 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:36.144072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:36.144657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.144929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.145198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.145822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:36.147755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:36.176305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1061/2145] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:36 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:36.180688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:36.191279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4780977-b69e-45f1-b7a1-0f58dba7b648 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:36.249437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:36.250214 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.250505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.250770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.251455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:36.253443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:36.275301 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1085/2146] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:36 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:36.279428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:36.290035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a78164-8237-4c4f-8c22-01c6a775a15f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:36.294528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:36.295111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.295419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.295689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.296311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:36.297896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:36.309690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1062/2147] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:36.313761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:36.324014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f37b67-9353-4c94-800d-da861c9697a5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:36.328932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507eef0-0f3d-4db9-823a-cd516d90a940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:36.329517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507eef0-0f3d-4db9-823a-cd516d90a940 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.329837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507eef0-0f3d-4db9-823a-cd516d90a940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.330092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507eef0-0f3d-4db9-823a-cd516d90a940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.330727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507eef0-0f3d-4db9-823a-cd516d90a940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:36.332342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507eef0-0f3d-4db9-823a-cd516d90a940 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:36.343594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1086/2148] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:36.347890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:36.358224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca31d8-3f71-4886-92a7-002a548adfa7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:36.362974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:36.363700 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.363983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.364254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] PATH_INFO: `/v3/projects/d780a60694e149c8b47943c3572a608f/users/64800f11861c42f1a635f22bb842ea7c/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.381542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d780a60694e149c8b47943c3572a608f, user_id=64800f11861c42f1a635f22bb842ea7c, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:36.384818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:36.397283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1063/2149] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:36 2022] PUT /identity/v3/projects/d780a60694e149c8b47943c3572a608f/users/64800f11861c42f1a635f22bb842ea7c/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:36.407591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5210-9b2e-40cf-9916-4615a196ded8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.407879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5210-9b2e-40cf-9916-4615a196ded8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.408149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5210-9b2e-40cf-9916-4615a196ded8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.438007 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a13c5210-9b2e-40cf-9916-4615a196ded8 None None] MFA Rules not processed for user `64800f11861c42f1a635f22bb842ea7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:36.439267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a13c5210-9b2e-40cf-9916-4615a196ded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:36.505335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1087/2150] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:36.516678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:36.518361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:36.563587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:36.580004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b9cd-114d-41f5-bb00-5609e84da0d3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:36.583570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574b88e-e7cc-497d-bef5-e519eb3b5ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:36.584165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574b88e-e7cc-497d-bef5-e519eb3b5ecb service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:36.584438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574b88e-e7cc-497d-bef5-e519eb3b5ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:36.584702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574b88e-e7cc-497d-bef5-e519eb3b5ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:36.593509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574b88e-e7cc-497d-bef5-e519eb3b5ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:36.595027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574b88e-e7cc-497d-bef5-e519eb3b5ecb service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:36.604143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1064/2151] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:09:39.664786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c5210-9b2e-40cf-9916-4615a196ded8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:39.713346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:39.729786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c5210-9b2e-40cf-9916-4615a196ded8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:39.734673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:39.735279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:39.735626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:39.736231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] PATH_INFO: `/v3/users/64800f11861c42f1a635f22bb842ea7c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:39.737169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64800f11861c42f1a635f22bb842ea7c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:39.739082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:39.789132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64800f11861c42f1a635f22bb842ea7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:39.789803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64800f11861c42f1a635f22bb842ea7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:39.796898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64800f11861c42f1a635f22bb842ea7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:39.797526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64800f11861c42f1a635f22bb842ea7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:39.810509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64800f11861c42f1a635f22bb842ea7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:39.824280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1088/2152] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:39 2022] DELETE /identity/v3/users/64800f11861c42f1a635f22bb842ea7c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:39.828724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:39.829212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:39.829328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:39.849102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:39.850394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:39.910569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1065/2153] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:39.920491 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:39.921949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:39.992912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5ded-c124-4254-84e2-4baf4adc6578 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:39.996678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99febc-339b-4367-a2e0-2c97efd774b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:39.997280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99febc-339b-4367-a2e0-2c97efd774b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:39.997562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99febc-339b-4367-a2e0-2c97efd774b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:39.997832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99febc-339b-4367-a2e0-2c97efd774b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.012862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99febc-339b-4367-a2e0-2c97efd774b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.014289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99febc-339b-4367-a2e0-2c97efd774b5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.024357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1089/2154] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:09:40.303508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.305665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:40.318416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110a01b-05f1-425e-921b-a87e92fb7738 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:40.323270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:40.323943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.324222 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.324514 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] PATH_INFO: `/v3/projects/d780a60694e149c8b47943c3572a608f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.325958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d780a60694e149c8b47943c3572a608f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.328048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.377098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd780a60694e149c8b47943c3572a608f', 'request_id': 'req-abd1f265-d994-40e8-a9f9-a9509753cb5d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:40.382797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd780a60694e149c8b47943c3572a608f', 'request_id': 'req-abd1f265-d994-40e8-a9f9-a9509753cb5d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:40.390100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd780a60694e149c8b47943c3572a608f', 'request_id': 'req-abd1f265-d994-40e8-a9f9-a9509753cb5d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:40.401891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] The token cache is being invalidate because project d780a60694e149c8b47943c3572a608f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:40.402176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:40.402831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-abd1f265-d994-40e8-a9f9-a9509753cb5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:40.404344 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1066/2155] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:40 2022] DELETE /identity/v3/projects/d780a60694e149c8b47943c3572a608f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:40.409433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533e771-48d1-4278-8ab9-c3c30ab1217a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.409773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533e771-48d1-4278-8ab9-c3c30ab1217a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.410061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533e771-48d1-4278-8ab9-c3c30ab1217a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.430703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d533e771-48d1-4278-8ab9-c3c30ab1217a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:40.431935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d533e771-48d1-4278-8ab9-c3c30ab1217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:40.496376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1090/2156] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:40.505524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577d68-ef1a-4f19-bae8-886fc245f8be None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.505805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577d68-ef1a-4f19-bae8-886fc245f8be None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.506079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577d68-ef1a-4f19-bae8-886fc245f8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.526137 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cd577d68-ef1a-4f19-bae8-886fc245f8be None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:40.527374 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cd577d68-ef1a-4f19-bae8-886fc245f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:40.535267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1067/2157] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:40.540310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533e771-48d1-4278-8ab9-c3c30ab1217a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.550391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533e771-48d1-4278-8ab9-c3c30ab1217a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:40.555728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:40.556347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.556646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.556934 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.557637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:09:40.557931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.559674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.561300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:40.578628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1091/2158] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:09:40.579807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd577d68-ef1a-4f19-bae8-886fc245f8be None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.589911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd577d68-ef1a-4f19-bae8-886fc245f8be None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:40.595459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:40.596156 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.596504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.596849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.597543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.599619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.627732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1068/2159] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:40 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:09:40.631959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.642848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbf87b-a20c-454a-92ff-a17dd06f9e80 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:40.697387 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:40.697995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.698169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.698438 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.699056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.701260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.722164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1092/2160] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:40 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:09:40.726774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.736754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368d9f2-410d-499e-873c-b98cd41df811 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:40.741716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:40.742408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.742688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.742956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.743632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.745251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.755142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1069/2161] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:40.758864 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.769359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72205986-ac2b-4b62-b960-3101d0fd9960 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:40.774567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a907826-e093-44f9-bb2c-0a16603ef67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:40.775162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a907826-e093-44f9-bb2c-0a16603ef67e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.775475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a907826-e093-44f9-bb2c-0a16603ef67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.775772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a907826-e093-44f9-bb2c-0a16603ef67e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.776405 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a907826-e093-44f9-bb2c-0a16603ef67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.778183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a907826-e093-44f9-bb2c-0a16603ef67e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.789666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1093/2162] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:40.794034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.805449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068048-f698-499c-ad2d-97dfac7bcd79 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:40.810508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:40.811169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.811470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.811783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] PATH_INFO: `/v3/projects/38222b0b5a8d4c268a88175af4be886f/users/d4a11aec8e294a43ba2d8162a14d04cf/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.826913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38222b0b5a8d4c268a88175af4be886f, user_id=d4a11aec8e294a43ba2d8162a14d04cf, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:40.829600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:40.842729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1070/2163] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:40 2022] PUT /identity/v3/projects/38222b0b5a8d4c268a88175af4be886f/users/d4a11aec8e294a43ba2d8162a14d04cf/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:40.853468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4858a3-0ac6-458b-823d-f372d394d9fc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:40.853792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4858a3-0ac6-458b-823d-f372d394d9fc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:40.854069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4858a3-0ac6-458b-823d-f372d394d9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:40.881932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad4858a3-0ac6-458b-823d-f372d394d9fc None None] MFA Rules not processed for user `d4a11aec8e294a43ba2d8162a14d04cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:40.883665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad4858a3-0ac6-458b-823d-f372d394d9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:40.947317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1094/2164] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:40.952061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:40.956372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:41.014493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448931e5-9eb7-4b85-83fa-1807d6662b3f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.019048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.019669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.019945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.020212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.020823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.022720 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.048650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1071/2165] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:40 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:41.052865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4858a3-0ac6-458b-823d-f372d394d9fc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.063199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4858a3-0ac6-458b-823d-f372d394d9fc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.117904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.118644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.118927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.119200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.119859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.121788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.130511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:41.144853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1095/2166] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:41 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:41.149519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.160623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef024-b091-4d84-b1cb-d9f926c44b3b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.165362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.165954 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.166259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.166530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.167165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.168827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.179704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1072/2167] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:41.183727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.194348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afa60a-455b-442f-8f25-d057ee45da8b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.200647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1123f3-6363-427a-a3e5-d41fa87337d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.201303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1123f3-6363-427a-a3e5-d41fa87337d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.201576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1123f3-6363-427a-a3e5-d41fa87337d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.201844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1123f3-6363-427a-a3e5-d41fa87337d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.202519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1123f3-6363-427a-a3e5-d41fa87337d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.204087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1123f3-6363-427a-a3e5-d41fa87337d2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.214370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1096/2168] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:41.218346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.228118 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62ccc9-efcb-4c43-94f3-2a862fce9610 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.232701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.233330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.233609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.233879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] PATH_INFO: `/v3/projects/7cccceee2bb54124b332eccf7c7cd83e/users/20783b38a4034dc19bf9138863d58ed1/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.248595 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cccceee2bb54124b332eccf7c7cd83e, user_id=20783b38a4034dc19bf9138863d58ed1, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.251266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.264275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1073/2169] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:41 2022] PUT /identity/v3/projects/7cccceee2bb54124b332eccf7c7cd83e/users/20783b38a4034dc19bf9138863d58ed1/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:41.274401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1115c-2a8f-43a8-bc94-680356bd7cd9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.274683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1115c-2a8f-43a8-bc94-680356bd7cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.274978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1115c-2a8f-43a8-bc94-680356bd7cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.301739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-36d1115c-2a8f-43a8-bc94-680356bd7cd9 None None] MFA Rules not processed for user `20783b38a4034dc19bf9138863d58ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:41.302921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-36d1115c-2a8f-43a8-bc94-680356bd7cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:41.364262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1097/2170] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:41.369450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.437403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49b09-4584-4d52-898b-6980c7b3c169 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.442284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.442880 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.443155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.443893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.444053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.446668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.476773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1074/2171] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:41 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:41.477922 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1115c-2a8f-43a8-bc94-680356bd7cd9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.488795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1115c-2a8f-43a8-bc94-680356bd7cd9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.545855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.546692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.546969 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.547265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.547908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.549840 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.571243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1098/2172] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:41 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:41.574810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.585244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc26a3-f29d-4480-91a0-27ff90978f3f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.590520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.591110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.591415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.591688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.592317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.593931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.606787 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1075/2173] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:09:41.610948 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.621529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da046-6be1-4cea-8866-4cc9d3171fa2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.627307 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dd60e-e116-41a5-97a7-8ad49ae1d53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.627955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd60e-e116-41a5-97a7-8ad49ae1d53f admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.628227 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd60e-e116-41a5-97a7-8ad49ae1d53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.628492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd60e-e116-41a5-97a7-8ad49ae1d53f admin admin] PATH_INFO: `/v3/projects/866bc5eb66ff448682aa41eae2326bac/users/6bc2fa1e51c349e1950fe0fd7b581792/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.646874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dd60e-e116-41a5-97a7-8ad49ae1d53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=866bc5eb66ff448682aa41eae2326bac, user_id=6bc2fa1e51c349e1950fe0fd7b581792, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.649602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dd60e-e116-41a5-97a7-8ad49ae1d53f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.664135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1099/2174] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:41 2022] PUT /identity/v3/projects/866bc5eb66ff448682aa41eae2326bac/users/6bc2fa1e51c349e1950fe0fd7b581792/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:41.668076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.678762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee24106-7536-4be8-bb54-e47f159d99f4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.712931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:41.756583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.758037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.758432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.758848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.759633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.761596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.763784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.764315 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.764415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.780228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1076/2175] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:09:41.781437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.787553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:41.789725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:41.793080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af339d-468a-4a84-8efe-2e815227ee61 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:41.797948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa579-4671-4ed4-bc68-7e801ef91895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:41.798502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa579-4671-4ed4-bc68-7e801ef91895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.798746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa579-4671-4ed4-bc68-7e801ef91895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.799071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa579-4671-4ed4-bc68-7e801ef91895 admin admin] PATH_INFO: `/v3/projects/866bc5eb66ff448682aa41eae2326bac/users/6bc2fa1e51c349e1950fe0fd7b581792/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.801267 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fa579-4671-4ed4-bc68-7e801ef91895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=866bc5eb66ff448682aa41eae2326bac, user_id=6bc2fa1e51c349e1950fe0fd7b581792, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:41.803997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fa579-4671-4ed4-bc68-7e801ef91895 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:41.826412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1077/2176] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:41 2022] PUT /identity/v3/projects/866bc5eb66ff448682aa41eae2326bac/users/6bc2fa1e51c349e1950fe0fd7b581792/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:41.839281 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:41.881946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1100/2177] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:09:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:41.898807 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:41.899676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:41.900175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:41.935694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] MFA Rules not processed for user `6bc2fa1e51c349e1950fe0fd7b581792`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:41.937693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:41.940759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:41.942029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:42.028265 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1078/2178] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:42.035675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:42.037069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:42.057146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e86ae2-a2c8-4fa8-a735-f17365b41f70 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:42.097397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0629e-0ceb-4add-b5b2-011fa5f90e5c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:42.101401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:42.101996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:42.102269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:42.102531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:42.117545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:42.118255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:42.118551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:42.118816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:42.157724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:42.161638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:42.164114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:42.174072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1079/2179] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:09:42.180772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:42.182361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:42.190558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1101/2180] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:09:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:09:43.705010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:43.714049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aefeab-df59-4a49-9f12-1565ab3b78f5 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:43.720401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:43.721020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:43.721356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:43.721636 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:43.802815 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:43.805678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:43.816602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1080/2181] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:44.633092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:44.645995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf48ad-6568-41ca-aca8-1a3e2ebd48dd service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:44.649835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3c42b-d7f0-4f8e-ac53-7efdaaa692c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:44.650432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c42b-d7f0-4f8e-ac53-7efdaaa692c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:44.650836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c42b-d7f0-4f8e-ac53-7efdaaa692c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:44.651124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c42b-d7f0-4f8e-ac53-7efdaaa692c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:44.664687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3c42b-d7f0-4f8e-ac53-7efdaaa692c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:44.666210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3c42b-d7f0-4f8e-ac53-7efdaaa692c9 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:44.676716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1102/2182] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:48.863327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:48.967680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf68bf-740c-4ada-aca0-680ce82bc459 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:48.972767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:48.973544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:48.973897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:48.974246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] PATH_INFO: `/v3/users/d4a11aec8e294a43ba2d8162a14d04cf` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:48.975525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a11aec8e294a43ba2d8162a14d04cf)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:48.978150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:49.054761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a11aec8e294a43ba2d8162a14d04cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.055494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a11aec8e294a43ba2d8162a14d04cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.065448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a11aec8e294a43ba2d8162a14d04cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.076098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a11aec8e294a43ba2d8162a14d04cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.076823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a11aec8e294a43ba2d8162a14d04cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.103247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1081/2183] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:48 2022] DELETE /identity/v3/users/d4a11aec8e294a43ba2d8162a14d04cf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:49.108752 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:49.110083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:49.110769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:49.136703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:49.138323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:49.152900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:49.233825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1103/2184] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:49.246166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:49.247888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:49.261591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:49.360498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4f464-451e-49ec-b5b6-57c39a1d5453 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:49.366974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:49.367677 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:49.367964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:49.368200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:49.380886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:49.381669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:49.392914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1082/2185] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:49.729469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:49.730995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:49.750683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a5120-2e72-415c-857c-e81c9665a5f9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:49.766082 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:49.766832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:49.767171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:49.767452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] PATH_INFO: `/v3/projects/38222b0b5a8d4c268a88175af4be886f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:49.768728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38222b0b5a8d4c268a88175af4be886f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:49.770803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:49.829362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38222b0b5a8d4c268a88175af4be886f', 'request_id': 'req-ba822895-afc8-4f78-945e-4713d795b1d1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.837601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38222b0b5a8d4c268a88175af4be886f', 'request_id': 'req-ba822895-afc8-4f78-945e-4713d795b1d1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.845469 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38222b0b5a8d4c268a88175af4be886f', 'request_id': 'req-ba822895-afc8-4f78-945e-4713d795b1d1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.862350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] The token cache is being invalidate because project 38222b0b5a8d4c268a88175af4be886f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:49.862654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.863629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:49.865423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1104/2186] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:49 2022] DELETE /identity/v3/projects/38222b0b5a8d4c268a88175af4be886f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:49.869509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:49.880569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d3735-3b51-41bc-83ad-c4aa9a0356b4 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:49.954352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:49.955065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:49.955297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:49.955572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] PATH_INFO: `/v3/users/20783b38a4034dc19bf9138863d58ed1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:49.956697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20783b38a4034dc19bf9138863d58ed1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:49.959062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:50.024701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20783b38a4034dc19bf9138863d58ed1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.025533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20783b38a4034dc19bf9138863d58ed1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.040048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20783b38a4034dc19bf9138863d58ed1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.049256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20783b38a4034dc19bf9138863d58ed1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.049965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20783b38a4034dc19bf9138863d58ed1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.071672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1083/2187] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:49 2022] DELETE /identity/v3/users/20783b38a4034dc19bf9138863d58ed1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:50.634421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:50.635970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:50.650616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba822895-afc8-4f78-945e-4713d795b1d1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:50.729062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:50.731303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:50.732401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:50.733166 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] PATH_INFO: `/v3/projects/7cccceee2bb54124b332eccf7c7cd83e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:50.740033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cccceee2bb54124b332eccf7c7cd83e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:50.743847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:50.792448 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:50.806201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cccceee2bb54124b332eccf7c7cd83e', 'request_id': 'req-590a269a-49ec-4be6-a1eb-166c4e8d404c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.818091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cccceee2bb54124b332eccf7c7cd83e', 'request_id': 'req-590a269a-49ec-4be6-a1eb-166c4e8d404c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.828309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cccceee2bb54124b332eccf7c7cd83e', 'request_id': 'req-590a269a-49ec-4be6-a1eb-166c4e8d404c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.845391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] The token cache is being invalidate because project 7cccceee2bb54124b332eccf7c7cd83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:50.846108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.848028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:50.853255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1105/2188] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:50 2022] DELETE /identity/v3/projects/7cccceee2bb54124b332eccf7c7cd83e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:50.854492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:50.856279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:50.870881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb143-2851-4d18-9e29-1f3016dc5928 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:50.914282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:50.972296 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:50.973081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:50.973358 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:50.973627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] PATH_INFO: `/v3/users/6bc2fa1e51c349e1950fe0fd7b581792` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:50.976729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc2fa1e51c349e1950fe0fd7b581792)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:50.993667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:51.039046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc2fa1e51c349e1950fe0fd7b581792'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.040867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc2fa1e51c349e1950fe0fd7b581792'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.048608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc2fa1e51c349e1950fe0fd7b581792'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.058877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc2fa1e51c349e1950fe0fd7b581792'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.060021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-175db7c1-d63e-42c8-9bcb-a9d3f51560d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc2fa1e51c349e1950fe0fd7b581792'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.080575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1084/2189] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:09:50 2022] DELETE /identity/v3/users/6bc2fa1e51c349e1950fe0fd7b581792 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:51.430434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:51.432353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:51.453455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a269a-49ec-4be6-a1eb-166c4e8d404c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:51.532713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:51.533378 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:51.533658 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:51.533935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] PATH_INFO: `/v3/projects/866bc5eb66ff448682aa41eae2326bac` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:51.537538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=866bc5eb66ff448682aa41eae2326bac)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:51.537538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:51.615427 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '866bc5eb66ff448682aa41eae2326bac', 'request_id': 'req-d0264338-d995-4f36-b73f-8f4dea768ca1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.623757 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '866bc5eb66ff448682aa41eae2326bac', 'request_id': 'req-d0264338-d995-4f36-b73f-8f4dea768ca1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.640135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '866bc5eb66ff448682aa41eae2326bac', 'request_id': 'req-d0264338-d995-4f36-b73f-8f4dea768ca1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.663645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] The token cache is being invalidate because project 866bc5eb66ff448682aa41eae2326bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:09:51.664180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.664888 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d0264338-d995-4f36-b73f-8f4dea768ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:09:51.666963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1106/2190] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:09:51 2022] DELETE /identity/v3/projects/866bc5eb66ff448682aa41eae2326bac => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:09:51.671578 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeffa6-3469-4833-afa5-26ea51ebb2e9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:51.671860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeffa6-3469-4833-afa5-26ea51ebb2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:51.672136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeffa6-3469-4833-afa5-26ea51ebb2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:51.719820 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-57eeffa6-3469-4833-afa5-26ea51ebb2e9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:51.721633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-57eeffa6-3469-4833-afa5-26ea51ebb2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:51.804496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1085/2191] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:51.812050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f52e1f-e172-4ba1-91e9-0a346de25025 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:51.812347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f52e1f-e172-4ba1-91e9-0a346de25025 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:51.812616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f52e1f-e172-4ba1-91e9-0a346de25025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:51.835457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-02f52e1f-e172-4ba1-91e9-0a346de25025 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:51.837454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-02f52e1f-e172-4ba1-91e9-0a346de25025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:51.845861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1107/2192] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:51.853293 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeffa6-3469-4833-afa5-26ea51ebb2e9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:51.863602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeffa6-3469-4833-afa5-26ea51ebb2e9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:51.869357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:51.869966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:51.870295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:51.870564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:51.873400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:09:51.873989 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:51.875321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:51.891548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1086/2193] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:09:51.894848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f52e1f-e172-4ba1-91e9-0a346de25025 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:51.904357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f52e1f-e172-4ba1-91e9-0a346de25025 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:51.909935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:51.910737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:51.911068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:51.911524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:51.912289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:51.914767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:51.952228 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1108/2194] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:09:51 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:51.956012 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:51.970975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9692a1-b42a-46bb-bef5-7acbc58514f3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:52.010922 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:52.057698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:52.058315 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:52.058597 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:52.058871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:52.059573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:52.072546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:52.104750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1087/2195] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:09:51 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:09:52.108953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:52.120044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b984f-4799-41bf-98fc-93591cd586d7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:52.124714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:52.125323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:52.125606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:52.125871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:52.126505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:52.128121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:52.140807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1109/2196] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:52.144651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:52.156555 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0707c8d-3c64-4fb1-aff6-30396f418e1d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:52.173353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42d735-1326-494f-8bbf-af0864c77d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:52.174231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d735-1326-494f-8bbf-af0864c77d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:52.174519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d735-1326-494f-8bbf-af0864c77d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:52.174832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d735-1326-494f-8bbf-af0864c77d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:52.175592 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42d735-1326-494f-8bbf-af0864c77d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:52.177313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42d735-1326-494f-8bbf-af0864c77d7f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:52.193793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1088/2197] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:09:52.195974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:52.208941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ff5aa-a858-419a-8256-9de4f23f85a9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:52.214929 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:52.223821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:52.224333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:52.224608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] PATH_INFO: `/v3/projects/cb49987b97454ad3a32b40bbbe6fc25d/users/78c2994104d84b4b855343f100903a05/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:52.239710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:09:52.244327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb49987b97454ad3a32b40bbbe6fc25d, user_id=78c2994104d84b4b855343f100903a05, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:52.247165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:52.264205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1110/2198] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:09:52 2022] PUT /identity/v3/projects/cb49987b97454ad3a32b40bbbe6fc25d/users/78c2994104d84b4b855343f100903a05/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:09:52.278746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe73ed-e0af-49c3-95c3-9333abac6861 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:52.279099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe73ed-e0af-49c3-95c3-9333abac6861 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:52.279473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe73ed-e0af-49c3-95c3-9333abac6861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:52.315891 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2ebe73ed-e0af-49c3-95c3-9333abac6861 None None] MFA Rules not processed for user `78c2994104d84b4b855343f100903a05`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:52.317309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2ebe73ed-e0af-49c3-95c3-9333abac6861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:52.393111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1089/2199] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:09:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:52.405442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:52.407127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:52.469205 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82296688-56e3-4454-9e2a-6aea92370694 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:52.472967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d246ca-858a-4f33-8364-3e4fd9c81cb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:52.473581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d246ca-858a-4f33-8364-3e4fd9c81cb2 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:52.473860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d246ca-858a-4f33-8364-3e4fd9c81cb2 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:52.474132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d246ca-858a-4f33-8364-3e4fd9c81cb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:52.483390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d246ca-858a-4f33-8364-3e4fd9c81cb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:52.484874 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d246ca-858a-4f33-8364-3e4fd9c81cb2 service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:52.495568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1111/2200] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:59.164245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0f68c-a12f-496c-a38d-b8145829edfb None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.164701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0f68c-a12f-496c-a38d-b8145829edfb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.164945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0f68c-a12f-496c-a38d-b8145829edfb None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.166606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1090/2201] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:09:59.172671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0c74d-2539-43d1-9054-a43cb8892863 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.173400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0c74d-2539-43d1-9054-a43cb8892863 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.173400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0c74d-2539-43d1-9054-a43cb8892863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.193865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-08d0c74d-2539-43d1-9054-a43cb8892863 None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:59.195063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-08d0c74d-2539-43d1-9054-a43cb8892863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:59.266540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1112/2202] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:09:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:09:59.279384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0f68c-a12f-496c-a38d-b8145829edfb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:59.281191 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-52b0f68c-a12f-496c-a38d-b8145829edfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:59.294138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0f68c-a12f-496c-a38d-b8145829edfb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:59.297619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14b0c2-8bb3-4626-a140-59f8b3b97923 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a3956bb98749e6bb19ee7f3465eac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:59.298203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b0c2-8bb3-4626-a140-59f8b3b97923 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.298473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b0c2-8bb3-4626-a140-59f8b3b97923 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.298738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b0c2-8bb3-4626-a140-59f8b3b97923 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.309026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14b0c2-8bb3-4626-a140-59f8b3b97923 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:59.310571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14b0c2-8bb3-4626-a140-59f8b3b97923 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:59.323542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1091/2203] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:09:59.394211 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2d6c1-5ea3-4e7f-8720-4bce01ff5f64 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.394581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2d6c1-5ea3-4e7f-8720-4bce01ff5f64 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.394921 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2d6c1-5ea3-4e7f-8720-4bce01ff5f64 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.396357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1113/2204] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:09:59.401793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473f9cf-6f77-41e2-8c01-37a09c60f053 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.402143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473f9cf-6f77-41e2-8c01-37a09c60f053 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.402481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473f9cf-6f77-41e2-8c01-37a09c60f053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.423487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9473f9cf-6f77-41e2-8c01-37a09c60f053 None None] MFA Rules not processed for user `56967a5531ea4e308af5b89c13524c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:59.424679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9473f9cf-6f77-41e2-8c01-37a09c60f053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:59.477764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1092/2205] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:09:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:59.489430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2d6c1-5ea3-4e7f-8720-4bce01ff5f64 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:59.491607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3c2d6c1-5ea3-4e7f-8720-4bce01ff5f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:59.504253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2d6c1-5ea3-4e7f-8720-4bce01ff5f64 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:59.508272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd14b53-9712-4e1e-8515-d5134bd9e3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:59.508845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd14b53-9712-4e1e-8515-d5134bd9e3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.509120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd14b53-9712-4e1e-8515-d5134bd9e3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.509384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd14b53-9712-4e1e-8515-d5134bd9e3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.516364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd14b53-9712-4e1e-8515-d5134bd9e3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:59.517846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd14b53-9712-4e1e-8515-d5134bd9e3f9 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:59.526119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1114/2206] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:09:59.608991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff5102-cafb-4a97-a213-8acb718723b4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.609300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff5102-cafb-4a97-a213-8acb718723b4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.609573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff5102-cafb-4a97-a213-8acb718723b4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.610908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1093/2207] 10.222.0.36 () {54 vars in 921 bytes} [Mon Oct 17 07:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:09:59.616373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb7cc6-dd79-44d4-ae42-6e8c75dc832d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.616660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb7cc6-dd79-44d4-ae42-6e8c75dc832d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.616945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb7cc6-dd79-44d4-ae42-6e8c75dc832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.636892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e2bb7cc6-dd79-44d4-ae42-6e8c75dc832d None None] MFA Rules not processed for user `74a91af405e24a0b8e9c893e1c299db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:09:59.638094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e2bb7cc6-dd79-44d4-ae42-6e8c75dc832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:09:59.647294 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1115/2208] 10.222.0.36 () {58 vars in 1018 bytes} [Mon Oct 17 07:09:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:09:59.657519 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff5102-cafb-4a97-a213-8acb718723b4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:09:59.669424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff5102-cafb-4a97-a213-8acb718723b4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:09:59.673149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:09:59.673781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:09:59.674056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:09:59.674328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:09:59.685603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:09:59.687016 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:09:59.697547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1094/2209] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:09:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:10:01.784245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fbc16-13f6-445a-9d4d-0ac21d890b68 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:01.784758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fbc16-13f6-445a-9d4d-0ac21d890b68 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:01.785046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fbc16-13f6-445a-9d4d-0ac21d890b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:01.804706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8c1fbc16-13f6-445a-9d4d-0ac21d890b68 None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:10:01.805893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8c1fbc16-13f6-445a-9d4d-0ac21d890b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:01.865608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1116/2210] 10.222.0.36 () {58 vars in 1009 bytes} [Mon Oct 17 07:10:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:10:01.886831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:10:01.889089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:01.944637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:10:01.995350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175762d5-fdd2-4b4a-802a-0eeb47cc3975 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:10:02.000647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:10:02.001257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:02.001538 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:02.001810 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:02.016113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:10:02.017729 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:10:02.027140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1095/2211] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:10:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 07:10:11.859373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:10:12.023664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=20 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:10:27.704215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fbc16-13f6-445a-9d4d-0ac21d890b68 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:10:27.713867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fbc16-13f6-445a-9d4d-0ac21d890b68 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:10:27.718185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe69b0-8332-480c-827a-c40414f171c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:10:27.718875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe69b0-8332-480c-827a-c40414f171c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:27.719272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe69b0-8332-480c-827a-c40414f171c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:27.719708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe69b0-8332-480c-827a-c40414f171c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:27.721908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3dfe69b0-8332-480c-827a-c40414f171c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:27.793021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe69b0-8332-480c-827a-c40414f171c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:10:27.794493 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe69b0-8332-480c-827a-c40414f171c5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:10:27.804736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1117/2212] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:10:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:10:29.120949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:10:29.187342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dca7-2da2-471a-afe2-b0fe37435724 service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:10:29.192071 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:10:29.192689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:29.193547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:29.193817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] PATH_INFO: `/v3/users/78c2994104d84b4b855343f100903a05` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:29.194920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c2994104d84b4b855343f100903a05)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:10:29.196887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:10:29.244605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c2994104d84b4b855343f100903a05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.245255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c2994104d84b4b855343f100903a05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.252098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c2994104d84b4b855343f100903a05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.260007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c2994104d84b4b855343f100903a05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.260657 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c2994104d84b4b855343f100903a05'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.277867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1096/2213] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:10:29 2022] DELETE /identity/v3/users/78c2994104d84b4b855343f100903a05 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:10:29.282316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:29.282805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:29.282903 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:29.303144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:10:29.304374 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:29.364184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1118/2214] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:10:29.374125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:10:29.375588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:29.437287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079466d-3e8d-456e-8539-e7e4d567983f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:10:29.442048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14522c99-e9f0-487c-8554-223719802154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:10:29.442791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14522c99-e9f0-487c-8554-223719802154 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:29.443666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14522c99-e9f0-487c-8554-223719802154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:29.444052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14522c99-e9f0-487c-8554-223719802154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:29.469478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14522c99-e9f0-487c-8554-223719802154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:10:29.471151 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14522c99-e9f0-487c-8554-223719802154 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:10:29.482106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1097/2215] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:10:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:10:29.796860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:10:29.798095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:29.810251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8dbd-3455-4f74-9e62-8c74a950411d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:10:29.815357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:10:29.816008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:29.816279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:29.816543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] PATH_INFO: `/v3/projects/cb49987b97454ad3a32b40bbbe6fc25d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:29.817668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb49987b97454ad3a32b40bbbe6fc25d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:10:29.820214 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:10:29.870817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb49987b97454ad3a32b40bbbe6fc25d', 'request_id': 'req-d718b497-8115-4c0c-aef9-c3d2ded24fb0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.876119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb49987b97454ad3a32b40bbbe6fc25d', 'request_id': 'req-d718b497-8115-4c0c-aef9-c3d2ded24fb0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.883515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb49987b97454ad3a32b40bbbe6fc25d', 'request_id': 'req-d718b497-8115-4c0c-aef9-c3d2ded24fb0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.895847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] The token cache is being invalidate because project cb49987b97454ad3a32b40bbbe6fc25d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:10:29.896684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.897293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:10:29.898916 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1119/2216] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:10:29 2022] DELETE /identity/v3/projects/cb49987b97454ad3a32b40bbbe6fc25d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:10:30.214539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95517f32-e17d-440e-b119-b6200fa327db None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:30.215068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95517f32-e17d-440e-b119-b6200fa327db None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:30.215068 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95517f32-e17d-440e-b119-b6200fa327db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:30.238769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-95517f32-e17d-440e-b119-b6200fa327db None None] MFA Rules not processed for user `96d71699c4c648529f5c6a9dd1ef6ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:10:30.240686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-95517f32-e17d-440e-b119-b6200fa327db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:30.260837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:10:30.307316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1098/2217] 10.222.0.36 () {58 vars in 967 bytes} [Mon Oct 17 07:10:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:10:30.321410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:10:30.322976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:30.345968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:10:30.407322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b497-8115-4c0c-aef9-c3d2ded24fb0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:10:30.413120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f4baa1481254765b8af7a253806f28f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:10:30.413738 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:30.414011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:30.414279 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:30.423268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:10:30.424856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:10:30.434723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1120/2218] 10.222.0.36 () {60 vars in 1384 bytes} [Mon Oct 17 07:10:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 07:10:40.300117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=15 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:10:40.431484 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=24 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:10:56.928402 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95517f32-e17d-440e-b119-b6200fa327db None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:10:56.929857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-95517f32-e17d-440e-b119-b6200fa327db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:56.991782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95517f32-e17d-440e-b119-b6200fa327db None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:10:56.995925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:10:56.996533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:10:56.996809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:10:56.997072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:10:56.998792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:10:57.062620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:10:57.064015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:10:57.074396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1099/2219] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:00.049553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:00.051108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:00.144196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f3826-4046-4e4c-a72e-63b87a2a64e7 service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:00.149945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:00.150583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.151054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.151232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] PATH_INFO: `/v3/users/1e37d9ed3751483ebe4141706f46e43a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.152628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e37d9ed3751483ebe4141706f46e43a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:00.154960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:00.214119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e37d9ed3751483ebe4141706f46e43a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.214837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e37d9ed3751483ebe4141706f46e43a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.221622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e37d9ed3751483ebe4141706f46e43a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.222250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e37d9ed3751483ebe4141706f46e43a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.234548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-94867564-d845-463a-a4e0-456f9d9bdb82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e37d9ed3751483ebe4141706f46e43a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.247443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1121/2220] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:00 2022] DELETE /identity/v3/users/1e37d9ed3751483ebe4141706f46e43a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:00.418706 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:00.481723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7b23b-ac2a-45d0-94c6-403914f015b3 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:00.535473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:00.536021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.536298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.536567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] PATH_INFO: `/v3/projects/934764e1a1394bfb91688060b594a58a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.537667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934764e1a1394bfb91688060b594a58a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:00.539607 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:00.551160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:00.585930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934764e1a1394bfb91688060b594a58a', 'request_id': 'req-210bb8ce-9953-4003-9c83-021733dc9c75'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.591822 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934764e1a1394bfb91688060b594a58a', 'request_id': 'req-210bb8ce-9953-4003-9c83-021733dc9c75'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.598702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934764e1a1394bfb91688060b594a58a', 'request_id': 'req-210bb8ce-9953-4003-9c83-021733dc9c75'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.609647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] The token cache is being invalidate because project 934764e1a1394bfb91688060b594a58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:00.609929 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.610541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-210bb8ce-9953-4003-9c83-021733dc9c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:00.612062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1100/2221] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:00 2022] DELETE /identity/v3/projects/934764e1a1394bfb91688060b594a58a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:00.616891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4679d4-c8a2-4a35-9435-1df0a56818fb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.617169 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4679d4-c8a2-4a35-9435-1df0a56818fb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.617439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4679d4-c8a2-4a35-9435-1df0a56818fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.674267 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bd4679d4-c8a2-4a35-9435-1df0a56818fb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:00.675485 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bd4679d4-c8a2-4a35-9435-1df0a56818fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:00.708565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:00.734794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1122/2222] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:00.742568 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059a81-e7b4-45de-b622-843c9b60d033 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.742846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059a81-e7b4-45de-b622-843c9b60d033 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.743111 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059a81-e7b4-45de-b622-843c9b60d033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.763308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d4059a81-e7b4-45de-b622-843c9b60d033 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:00.764482 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d4059a81-e7b4-45de-b622-843c9b60d033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:00.772197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1101/2223] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:00.776139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4679d4-c8a2-4a35-9435-1df0a56818fb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:00.785366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4679d4-c8a2-4a35-9435-1df0a56818fb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:00.789854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:00.790375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.790647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.790914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.791578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:00.791899 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:00.793467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:00.804820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1123/2224] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:00.809366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4059a81-e7b4-45de-b622-843c9b60d033 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:00.818423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4059a81-e7b4-45de-b622-843c9b60d033 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:00.823045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:00.824168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.824443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.824714 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.825594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:00.827528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:00.854286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1102/2225] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:00 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:00.858350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:00.868510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3e43-4f04-4690-bf8c-b54fb27b1aff admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:00.926786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:00.927865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.928232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.928616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.929400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:00.933033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:00.963592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1124/2226] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:00 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:00.968162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:00.978961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e66229-e21d-4f5c-9f67-f6188e29c9da admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:00.984243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:00.985022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:00.985302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:00.985628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:00.988648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:00.991861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.007360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1103/2227] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:01.009381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.021905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539d809-8100-40bb-8477-3607ea9f895a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.027141 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58264f-75a5-4926-b53d-6802643c31c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.027795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58264f-75a5-4926-b53d-6802643c31c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.028074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58264f-75a5-4926-b53d-6802643c31c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.028342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58264f-75a5-4926-b53d-6802643c31c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.029197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58264f-75a5-4926-b53d-6802643c31c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.031542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58264f-75a5-4926-b53d-6802643c31c1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.048942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1125/2228] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:01.050553 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.063691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b64e-bb1d-448f-88e3-2e295a734cd5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.069992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.070676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.071086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.071419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] PATH_INFO: `/v3/projects/f16ecaf7f27b4290801b437e1542dc93/users/cf4bbdbdc53a47d0a84b11e83237ed8b/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.088743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16ecaf7f27b4290801b437e1542dc93, user_id=cf4bbdbdc53a47d0a84b11e83237ed8b, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.091450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.105166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1104/2229] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:01 2022] PUT /identity/v3/projects/f16ecaf7f27b4290801b437e1542dc93/users/cf4bbdbdc53a47d0a84b11e83237ed8b/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:01.116984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e4c8e-dcff-495d-93cb-cf4c7dc53fbb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.117507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e4c8e-dcff-495d-93cb-cf4c7dc53fbb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.117507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e4c8e-dcff-495d-93cb-cf4c7dc53fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.145657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-983e4c8e-dcff-495d-93cb-cf4c7dc53fbb None None] MFA Rules not processed for user `cf4bbdbdc53a47d0a84b11e83237ed8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:01.146855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-983e4c8e-dcff-495d-93cb-cf4c7dc53fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:01.210958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1126/2230] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:01.215502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.276674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a6789-2e5f-44dc-8cb5-19cdb652657d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.281262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.281877 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.282147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.282412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.283036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.284949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.314338 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1105/2231] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:01 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:01.318444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e4c8e-dcff-495d-93cb-cf4c7dc53fbb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.329229 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e4c8e-dcff-495d-93cb-cf4c7dc53fbb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.362546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:01.387515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.388121 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.388401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.388672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.389296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.391229 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.412705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1127/2232] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:01 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:01.416667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.427436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3447d9d-e48a-402e-afd1-695033ada81a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.431960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.432570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.432844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.433113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.433741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.435369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.445593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1106/2233] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:01.449655 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.459754 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b381df-08cf-425f-8f06-60111cf33101 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.464978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.465612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.465891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.466164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] PATH_INFO: `/v3/projects/391e89d345434a62a12996793830de29/users/216a7f551e004a0398dc14e2c0125d13/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.481090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391e89d345434a62a12996793830de29, user_id=216a7f551e004a0398dc14e2c0125d13, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.483802 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.496408 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1128/2234] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:01 2022] PUT /identity/v3/projects/391e89d345434a62a12996793830de29/users/216a7f551e004a0398dc14e2c0125d13/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:01.500349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.510037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120fc79-c31f-4d7b-8cff-1d223e1426ce admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.517622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:01.567162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886626f-f109-4491-821b-59c1ab7420d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.568010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886626f-f109-4491-821b-59c1ab7420d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.568287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886626f-f109-4491-821b-59c1ab7420d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.568552 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886626f-f109-4491-821b-59c1ab7420d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.569179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886626f-f109-4491-821b-59c1ab7420d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.570775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886626f-f109-4491-821b-59c1ab7420d9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.580606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1107/2235] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:01.584831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.595396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba69e9-d19f-4cdb-b11b-202b595992f1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.599916 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.600547 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.600825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.601097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] PATH_INFO: `/v3/projects/391e89d345434a62a12996793830de29/users/216a7f551e004a0398dc14e2c0125d13/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.602975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391e89d345434a62a12996793830de29, user_id=216a7f551e004a0398dc14e2c0125d13, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.606192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.616164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1129/2236] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:01 2022] PUT /identity/v3/projects/391e89d345434a62a12996793830de29/users/216a7f551e004a0398dc14e2c0125d13/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:01.625897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd3832-4fe5-401b-bb67-19e8dd765ad2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.626199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd3832-4fe5-401b-bb67-19e8dd765ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.626453 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd3832-4fe5-401b-bb67-19e8dd765ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.653785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-31cd3832-4fe5-401b-bb67-19e8dd765ad2 None None] MFA Rules not processed for user `216a7f551e004a0398dc14e2c0125d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:01.654967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-31cd3832-4fe5-401b-bb67-19e8dd765ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:01.723442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1108/2237] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:01.735451 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:01.737159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:01.808480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2932a6-e60e-4705-a8d5-10b26216d275 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:01.812071 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:01.812658 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:01.812932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:01.813207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:01.821957 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:01.823393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:01.833170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1130/2238] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:02.215020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd3832-4fe5-401b-bb67-19e8dd765ad2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:02.228452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:02.230286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:02.281218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd3832-4fe5-401b-bb67-19e8dd765ad2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:02.285708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:02.286326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:02.286601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:02.286872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] PATH_INFO: `/v3/users/cf4bbdbdc53a47d0a84b11e83237ed8b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:02.287968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505d4c7-699a-48b5-a63d-9134213cc67c service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:02.288091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf4bbdbdc53a47d0a84b11e83237ed8b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:02.289907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:02.291662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f581f-2c91-4eda-9341-f5f94410f134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:02.292288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f581f-2c91-4eda-9341-f5f94410f134 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:02.292577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f581f-2c91-4eda-9341-f5f94410f134 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:02.292847 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f581f-2c91-4eda-9341-f5f94410f134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:02.295086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-502f581f-2c91-4eda-9341-f5f94410f134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:02.307132 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f581f-2c91-4eda-9341-f5f94410f134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:02.308588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f581f-2c91-4eda-9341-f5f94410f134 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:02.318855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1131/2239] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:02.333986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:02.349348 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf4bbdbdc53a47d0a84b11e83237ed8b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.350401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf4bbdbdc53a47d0a84b11e83237ed8b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.363126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf4bbdbdc53a47d0a84b11e83237ed8b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.373389 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf4bbdbdc53a47d0a84b11e83237ed8b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.374363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf4bbdbdc53a47d0a84b11e83237ed8b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.395313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1109/2240] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:02 2022] DELETE /identity/v3/users/cf4bbdbdc53a47d0a84b11e83237ed8b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:02.399822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:02.400108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:02.400389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:02.420445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:02.421645 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:02.433381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:02.482283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1132/2241] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:02.492034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:02.494401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:02.552920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874ec7-da2c-4e79-8852-e20ed400f8c3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:02.556829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:02.557412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:02.557690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:02.557962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:02.567075 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:02.568542 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:02.578786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1110/2242] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:02.870883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:02.872986 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:02.887181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1f738-0694-444c-9f34-34c7010a1881 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:02.893692 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:02.894875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:02.894875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:02.895337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] PATH_INFO: `/v3/projects/f16ecaf7f27b4290801b437e1542dc93` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:02.896549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16ecaf7f27b4290801b437e1542dc93)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:02.898538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:02.952997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16ecaf7f27b4290801b437e1542dc93', 'request_id': 'req-8650b34f-ae9e-4643-8af8-e4614c40df72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.958746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16ecaf7f27b4290801b437e1542dc93', 'request_id': 'req-8650b34f-ae9e-4643-8af8-e4614c40df72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.965681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16ecaf7f27b4290801b437e1542dc93', 'request_id': 'req-8650b34f-ae9e-4643-8af8-e4614c40df72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.979646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] The token cache is being invalidate because project f16ecaf7f27b4290801b437e1542dc93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:02.980009 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.980713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:02.982321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1133/2243] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:02 2022] DELETE /identity/v3/projects/f16ecaf7f27b4290801b437e1542dc93 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:02.986562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:02.997146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea5809-6338-43f3-84d0-0aa3ad580da2 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:03.062610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:03.063282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:03.063564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:03.063837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] PATH_INFO: `/v3/users/216a7f551e004a0398dc14e2c0125d13` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:03.064961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=216a7f551e004a0398dc14e2c0125d13)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:03.067110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:03.124262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216a7f551e004a0398dc14e2c0125d13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.124858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216a7f551e004a0398dc14e2c0125d13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.132185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216a7f551e004a0398dc14e2c0125d13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.133907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:03.141199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216a7f551e004a0398dc14e2c0125d13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.142360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aaa561bd-2ec4-4133-ac58-7039aac33f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216a7f551e004a0398dc14e2c0125d13'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.170143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1111/2244] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:02 2022] DELETE /identity/v3/users/216a7f551e004a0398dc14e2c0125d13 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:03.470771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:03.473043 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:03.518022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650b34f-ae9e-4643-8af8-e4614c40df72 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:03.593117 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:03.593862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:03.594230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:03.594574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] PATH_INFO: `/v3/projects/391e89d345434a62a12996793830de29` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:03.595906 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=391e89d345434a62a12996793830de29)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:03.598215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:03.650090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:03.659996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391e89d345434a62a12996793830de29', 'request_id': 'req-e236e962-adf3-4318-88cd-25eefb309ee1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.666217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391e89d345434a62a12996793830de29', 'request_id': 'req-e236e962-adf3-4318-88cd-25eefb309ee1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.674879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391e89d345434a62a12996793830de29', 'request_id': 'req-e236e962-adf3-4318-88cd-25eefb309ee1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.688100 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] The token cache is being invalidate because project 391e89d345434a62a12996793830de29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:03.688434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.689015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e236e962-adf3-4318-88cd-25eefb309ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:03.690534 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1134/2245] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:03 2022] DELETE /identity/v3/projects/391e89d345434a62a12996793830de29 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:03.696811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2631-4ed5-4c30-a105-fcd5efe80dad None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:03.697307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2631-4ed5-4c30-a105-fcd5efe80dad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:03.697567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2631-4ed5-4c30-a105-fcd5efe80dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:03.720488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-022f2631-4ed5-4c30-a105-fcd5efe80dad None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:03.721848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-022f2631-4ed5-4c30-a105-fcd5efe80dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:03.791313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1112/2246] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:03.800696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e745a-924f-421c-b85a-cd4be0072dd4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:03.800981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e745a-924f-421c-b85a-cd4be0072dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:03.801254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e745a-924f-421c-b85a-cd4be0072dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:03.828914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0d9e745a-924f-421c-b85a-cd4be0072dd4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:03.830263 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0d9e745a-924f-421c-b85a-cd4be0072dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:03.839098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1135/2247] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:03.843944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2631-4ed5-4c30-a105-fcd5efe80dad None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:03.853671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2631-4ed5-4c30-a105-fcd5efe80dad None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:03.859070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:03.859804 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:03.860081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:03.860534 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:03.861248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:03.861617 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:03.863327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:03.875802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1113/2248] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:03.880021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e745a-924f-421c-b85a-cd4be0072dd4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:03.890035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e745a-924f-421c-b85a-cd4be0072dd4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:03.894837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:03.895400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:03.895678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:03.895945 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:03.896579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:03.898463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:03.931150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1136/2249] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:03 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:03.935142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:03.945527 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1872454-2972-4002-8817-619ff236fa97 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.023045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.024042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.024042 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.024273 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.024865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.027072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.060782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1114/2250] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:03 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:04.067616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.093904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e1e9e-f1e7-416a-86f6-3dd8503166ea admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.098882 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.099951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.100313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.100657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.101576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.103531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.116134 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1137/2251] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:04.119800 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.131589 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f821fc-8cf5-4840-8bfa-317a52e1316d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.138724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf730342-2a7d-4179-a63d-25da95effa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.139746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730342-2a7d-4179-a63d-25da95effa44 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.140207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730342-2a7d-4179-a63d-25da95effa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.140619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730342-2a7d-4179-a63d-25da95effa44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.141408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf730342-2a7d-4179-a63d-25da95effa44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.143656 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf730342-2a7d-4179-a63d-25da95effa44 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.172382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1115/2252] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:04.176739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.187653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9fc6-19d3-4569-84d2-b8e81fe46f69 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.192579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.193401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.193690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.193965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] PATH_INFO: `/v3/projects/f1da8d9162a846c98ff9d9f9adf3cd84/users/1994b76d859646d18e2f84e100a8d03d/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.216524 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1da8d9162a846c98ff9d9f9adf3cd84, user_id=1994b76d859646d18e2f84e100a8d03d, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.219171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.243349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1138/2253] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:04 2022] PUT /identity/v3/projects/f1da8d9162a846c98ff9d9f9adf3cd84/users/1994b76d859646d18e2f84e100a8d03d/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:04.253732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec620717-d065-4c1b-99e0-6f96aa5c7fde None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.254073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec620717-d065-4c1b-99e0-6f96aa5c7fde None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.254260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec620717-d065-4c1b-99e0-6f96aa5c7fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.288498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ec620717-d065-4c1b-99e0-6f96aa5c7fde None None] MFA Rules not processed for user `1994b76d859646d18e2f84e100a8d03d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:04.289734 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ec620717-d065-4c1b-99e0-6f96aa5c7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:04.346925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:04.408788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1116/2254] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:04.413420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.499138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d71b4-a932-4a78-a4c7-8a75c9255b1d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.508363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.509025 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.509396 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.509759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.510510 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.512579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.553333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1139/2255] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:04 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:04.554842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec620717-d065-4c1b-99e0-6f96aa5c7fde None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.568378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec620717-d065-4c1b-99e0-6f96aa5c7fde None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.653419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.654357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.655072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.655564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.656309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.658743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.687184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1117/2256] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:04 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:04.688582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.702619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df68380-a34a-4fe7-aa97-002d2b4aacee admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.710305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.711012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.711184 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.711479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.712118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.713732 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.725559 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1140/2257] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:04.730027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.743384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364b920-03cb-486e-b978-e4c6ef5a4374 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.749567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.750622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.750622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.750811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] PATH_INFO: `/v3/projects/8ea2a9c732fe404a8126289b912c933b/users/de525401a3ff49db9680269a4c0388ca/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.770905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea2a9c732fe404a8126289b912c933b, user_id=de525401a3ff49db9680269a4c0388ca, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.773996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.791655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1118/2258] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:04 2022] PUT /identity/v3/projects/8ea2a9c732fe404a8126289b912c933b/users/de525401a3ff49db9680269a4c0388ca/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:04.795979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.806942 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f09ed9-9b7a-4640-b85a-4351e722940b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.854304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:04.877202 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414b2f9-924e-456f-9310-5d8077ddcdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.877817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414b2f9-924e-456f-9310-5d8077ddcdbc admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.878095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414b2f9-924e-456f-9310-5d8077ddcdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.878365 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414b2f9-924e-456f-9310-5d8077ddcdbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.879258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414b2f9-924e-456f-9310-5d8077ddcdbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.880837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414b2f9-924e-456f-9310-5d8077ddcdbc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.898471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1141/2259] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:04.902789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:04.913890 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628dfe-f499-4d52-a94f-05bf864908d0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:04.918637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:04.919938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.920283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.920567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] PATH_INFO: `/v3/projects/8ea2a9c732fe404a8126289b912c933b/users/de525401a3ff49db9680269a4c0388ca/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.922481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea2a9c732fe404a8126289b912c933b, user_id=de525401a3ff49db9680269a4c0388ca, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:04.925187 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:04.940681 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1119/2260] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:04 2022] PUT /identity/v3/projects/8ea2a9c732fe404a8126289b912c933b/users/de525401a3ff49db9680269a4c0388ca/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:04.951101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5713902-d53d-4d89-b326-297fb669d325 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:04.951444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5713902-d53d-4d89-b326-297fb669d325 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:04.951695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5713902-d53d-4d89-b326-297fb669d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:04.981164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f5713902-d53d-4d89-b326-297fb669d325 None None] MFA Rules not processed for user `de525401a3ff49db9680269a4c0388ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:04.982924 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f5713902-d53d-4d89-b326-297fb669d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:05.052900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1142/2261] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:05.064212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:05.065924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:05.128793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c28d4-2e0d-4c64-ae13-dfbd6bd8a471 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:05.132806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945d880-d0b4-43f1-b915-9deff21dbef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:05.133588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945d880-d0b4-43f1-b915-9deff21dbef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:05.133926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945d880-d0b4-43f1-b915-9deff21dbef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:05.134302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945d880-d0b4-43f1-b915-9deff21dbef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:05.144931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945d880-d0b4-43f1-b915-9deff21dbef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:05.146319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945d880-d0b4-43f1-b915-9deff21dbef5 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:05.158065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1120/2262] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:05.693970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5713902-d53d-4d89-b326-297fb669d325 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:05.782953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5713902-d53d-4d89-b326-297fb669d325 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:05.788858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:05.789702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:05.790052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:05.790413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] PATH_INFO: `/v3/users/1994b76d859646d18e2f84e100a8d03d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:05.792349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1994b76d859646d18e2f84e100a8d03d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:05.795516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:05.849262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1994b76d859646d18e2f84e100a8d03d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:05.850377 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1994b76d859646d18e2f84e100a8d03d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:05.862561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1994b76d859646d18e2f84e100a8d03d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:05.863475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1994b76d859646d18e2f84e100a8d03d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:05.871514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:05.879419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1994b76d859646d18e2f84e100a8d03d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:05.903844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1143/2263] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:05 2022] DELETE /identity/v3/users/1994b76d859646d18e2f84e100a8d03d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:05.905840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:05.906321 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:05.906446 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:05.940523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:05.940523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:05.964334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:06.009125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1121/2264] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:06.019723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.021230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:06.068060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.070603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:06.088503 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a23102-4d9e-4234-813f-7e6a6e5fec24 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.093818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.094990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.095104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.095368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] PATH_INFO: `/v3/users/51b2f292cea24a21928e2adcef0c3786` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.100124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b2f292cea24a21928e2adcef0c3786)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:06.102312 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:06.111884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ed7d-4b58-4554-ba3f-67b47371f26d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.115431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.116032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.116305 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.116570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.126918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:06.128400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:06.139183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1144/2265] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:06.164361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b2f292cea24a21928e2adcef0c3786'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.165041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b2f292cea24a21928e2adcef0c3786'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.172854 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b2f292cea24a21928e2adcef0c3786'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.179865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b2f292cea24a21928e2adcef0c3786'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.180591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b2f292cea24a21928e2adcef0c3786'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.199421 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1122/2266] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:06 2022] DELETE /identity/v3/users/51b2f292cea24a21928e2adcef0c3786 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:06.457455 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.467893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98248-70d1-42cd-bb00-7144ff794cd4 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.485108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.487135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:06.550259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.550259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.550599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.550706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] PATH_INFO: `/v3/projects/f992a8ccca8d487abd91dc6d0baaf3bc` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.551801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f992a8ccca8d487abd91dc6d0baaf3bc)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:06.553698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:06.580336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39cc06-28c0-4a3a-9ffd-907219008fa8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.585511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.586139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.586411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.586679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] PATH_INFO: `/v3/projects/f1da8d9162a846c98ff9d9f9adf3cd84` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.587831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1da8d9162a846c98ff9d9f9adf3cd84)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:06.589741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:06.600857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f992a8ccca8d487abd91dc6d0baaf3bc', 'request_id': 'req-fed30428-6ae6-46ea-a9cd-790a1d257e5c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.607068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f992a8ccca8d487abd91dc6d0baaf3bc', 'request_id': 'req-fed30428-6ae6-46ea-a9cd-790a1d257e5c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.613461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f992a8ccca8d487abd91dc6d0baaf3bc', 'request_id': 'req-fed30428-6ae6-46ea-a9cd-790a1d257e5c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.626725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] The token cache is being invalidate because project f992a8ccca8d487abd91dc6d0baaf3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:06.627005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.627830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fed30428-6ae6-46ea-a9cd-790a1d257e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.629444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1145/2267] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:06 2022] DELETE /identity/v3/projects/f992a8ccca8d487abd91dc6d0baaf3bc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:06.631484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:06.634476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58377-9bbf-4eba-aee9-6b4a5b1bc543 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.634759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58377-9bbf-4eba-aee9-6b4a5b1bc543 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.635031 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58377-9bbf-4eba-aee9-6b4a5b1bc543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.639121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1da8d9162a846c98ff9d9f9adf3cd84', 'request_id': 'req-20ad4720-8425-4960-9bc9-1f3e5b9244ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.644965 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1da8d9162a846c98ff9d9f9adf3cd84', 'request_id': 'req-20ad4720-8425-4960-9bc9-1f3e5b9244ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.654287 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1da8d9162a846c98ff9d9f9adf3cd84', 'request_id': 'req-20ad4720-8425-4960-9bc9-1f3e5b9244ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.655171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-19d58377-9bbf-4eba-aee9-6b4a5b1bc543 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:06.656413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-19d58377-9bbf-4eba-aee9-6b4a5b1bc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:06.665821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:06.667737 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] The token cache is being invalidate because project f1da8d9162a846c98ff9d9f9adf3cd84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:06.668021 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.668733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.670840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1123/2268] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:06 2022] DELETE /identity/v3/projects/f1da8d9162a846c98ff9d9f9adf3cd84 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:06.674766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.676504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:06.721778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1146/2269] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:06.729606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544b84f-daa9-4802-a3dc-bfa7b915ab7c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.729939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544b84f-daa9-4802-a3dc-bfa7b915ab7c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.730343 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544b84f-daa9-4802-a3dc-bfa7b915ab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.740320 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad4720-8425-4960-9bc9-1f3e5b9244ea admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.744767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.745381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.745652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.745919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] PATH_INFO: `/v3/users/de525401a3ff49db9680269a4c0388ca` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.747063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de525401a3ff49db9680269a4c0388ca)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:06.749011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:06.751054 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2544b84f-daa9-4802-a3dc-bfa7b915ab7c None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:06.752261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2544b84f-daa9-4802-a3dc-bfa7b915ab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:06.760249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1147/2270] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:06.764466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544b84f-daa9-4802-a3dc-bfa7b915ab7c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.773249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544b84f-daa9-4802-a3dc-bfa7b915ab7c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.778973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.779607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.779887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.780162 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.780799 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:06.781166 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:06.782898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:06.795634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1148/2271] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:06.799830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.799994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de525401a3ff49db9680269a4c0388ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.800582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de525401a3ff49db9680269a4c0388ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.807926 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de525401a3ff49db9680269a4c0388ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.809172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8934e-9d27-44f8-9a46-35685ee5ec2d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.814157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.814854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.815135 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:06.815681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:06.816107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de525401a3ff49db9680269a4c0388ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.816379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:06.816829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de525401a3ff49db9680269a4c0388ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:06.818364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:06.834827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1124/2272] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:06 2022] DELETE /identity/v3/users/de525401a3ff49db9680269a4c0388ca => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:06.852464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1149/2273] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:06 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:06.853571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:06.855133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:06.905412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:06.922608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a10b9-ef94-4991-91e2-601793035ed6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:06.998544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:06.999312 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:06.999740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.000106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.000781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.003573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.029190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1125/2274] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:06 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:07.033326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.047995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af0367-cc65-48ea-88dd-71cb32ba0cb3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.054296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.063516 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.067019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.067406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.068138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.069838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.080592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1150/2275] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:07.084567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.094483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b5b5-a10b-470f-a169-823919118b06 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.100894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.101650 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.102138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.102504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.103145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.104943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.116060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1126/2276] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:07.120355 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.124938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.128041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:07.132091 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a54e-8791-4283-93a4-2cfdf8fd911b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.136766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c905a59-cae1-4597-af5c-0049333f10c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.137505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c905a59-cae1-4597-af5c-0049333f10c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.137876 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c905a59-cae1-4597-af5c-0049333f10c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.138215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c905a59-cae1-4597-af5c-0049333f10c5 admin admin] PATH_INFO: `/v3/projects/6e2a9c2575114667bd8ed552b3725ed7/users/99a16caa037b4cf6b1e62a5779e04e56/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.148081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813f68a-008c-4c7e-b6e6-f26454e060c6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.160793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.161011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c905a59-cae1-4597-af5c-0049333f10c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2a9c2575114667bd8ed552b3725ed7, user_id=99a16caa037b4cf6b1e62a5779e04e56, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.161477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.161742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.162018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] PATH_INFO: `/v3/projects/8ea2a9c732fe404a8126289b912c933b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.163138 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea2a9c732fe404a8126289b912c933b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.164020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c905a59-cae1-4597-af5c-0049333f10c5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.165083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.178632 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1151/2277] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:07 2022] PUT /identity/v3/projects/6e2a9c2575114667bd8ed552b3725ed7/users/99a16caa037b4cf6b1e62a5779e04e56/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:07.189543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33903f52-02e8-4070-b08b-65382765d516 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.189829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33903f52-02e8-4070-b08b-65382765d516 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.190101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33903f52-02e8-4070-b08b-65382765d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.215173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea2a9c732fe404a8126289b912c933b', 'request_id': 'req-a688071c-290c-45da-93e6-7467ea63c422'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:07.219621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-33903f52-02e8-4070-b08b-65382765d516 None None] MFA Rules not processed for user `99a16caa037b4cf6b1e62a5779e04e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:07.220830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-33903f52-02e8-4070-b08b-65382765d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:07.220973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea2a9c732fe404a8126289b912c933b', 'request_id': 'req-a688071c-290c-45da-93e6-7467ea63c422'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:07.229319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea2a9c732fe404a8126289b912c933b', 'request_id': 'req-a688071c-290c-45da-93e6-7467ea63c422'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:07.241912 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] The token cache is being invalidate because project 8ea2a9c732fe404a8126289b912c933b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:07.242197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:07.242858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a688071c-290c-45da-93e6-7467ea63c422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:07.244946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1127/2278] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:07 2022] DELETE /identity/v3/projects/8ea2a9c732fe404a8126289b912c933b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:07.249988 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405956a8-c228-473b-938f-0f5a401998fc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.250272 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405956a8-c228-473b-938f-0f5a401998fc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.250549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405956a8-c228-473b-938f-0f5a401998fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.269164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-405956a8-c228-473b-938f-0f5a401998fc None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:07.270360 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-405956a8-c228-473b-938f-0f5a401998fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:07.292372 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1152/2279] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:07.300696 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33903f52-02e8-4070-b08b-65382765d516 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.302118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-33903f52-02e8-4070-b08b-65382765d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:07.321736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:07.335554 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1128/2280] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:07.343108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:07.343572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbf804-136b-4c44-813e-d7f447200b01 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.343852 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbf804-136b-4c44-813e-d7f447200b01 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.344122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbf804-136b-4c44-813e-d7f447200b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.364227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d8fbf804-136b-4c44-813e-d7f447200b01 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:07.365403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d8fbf804-136b-4c44-813e-d7f447200b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:07.373508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1129/2281] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:07.377562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbf804-136b-4c44-813e-d7f447200b01 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.386186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbf804-136b-4c44-813e-d7f447200b01 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.390727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.391353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.391618 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.391885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.392506 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:07.392805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.394390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.406802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1130/2282] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:07.410962 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.416759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33903f52-02e8-4070-b08b-65382765d516 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.420882 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910aef-922f-48a6-9381-5cc1e871f887 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.422815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.423443 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.423723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.423994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.425541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.426179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.426524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.426794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.427464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.429342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.431776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.433332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.442080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1153/2283] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:07.459247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1131/2284] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:07.462309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.474461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534001-d77e-4a4f-b5ae-97c436e818ae service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.532460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.533064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.533340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.533601 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.534221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.536167 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.560095 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1154/2285] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:07.564245 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.574432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c00d9-4579-467d-a2f4-21cb9c72955d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.580171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.580750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.581040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.581294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.582294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.584305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.595864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1132/2286] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:07.600407 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.611740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7111075-d5dc-4e36-87d1-c6f719d50917 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.616955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452a525-c348-4757-be3f-30bae90fdf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.617548 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a525-c348-4757-be3f-30bae90fdf92 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.617824 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a525-c348-4757-be3f-30bae90fdf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.618097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a525-c348-4757-be3f-30bae90fdf92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.618720 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452a525-c348-4757-be3f-30bae90fdf92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.620364 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452a525-c348-4757-be3f-30bae90fdf92 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.630967 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1155/2287] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:07.636434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.646143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245d93a-0a30-4fb1-a3e0-0fd551bad817 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.650707 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.651381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.651654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.651925 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] PATH_INFO: `/v3/projects/3c1a6fa0c17348e6a8aea03c0e787d2a/users/940ad712d03947a6b00fae12944f78ff/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.669540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1a6fa0c17348e6a8aea03c0e787d2a, user_id=940ad712d03947a6b00fae12944f78ff, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.672254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.685499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1133/2288] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:07 2022] PUT /identity/v3/projects/3c1a6fa0c17348e6a8aea03c0e787d2a/users/940ad712d03947a6b00fae12944f78ff/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:07.696768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741821-bd8d-44b1-a8f4-7a9558122889 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.697047 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741821-bd8d-44b1-a8f4-7a9558122889 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.697366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741821-bd8d-44b1-a8f4-7a9558122889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.727432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-21741821-bd8d-44b1-a8f4-7a9558122889 None None] MFA Rules not processed for user `940ad712d03947a6b00fae12944f78ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:07.728729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-21741821-bd8d-44b1-a8f4-7a9558122889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:07.796262 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:07.809425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1156/2289] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:07.814043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.877721 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5732d5b-a50b-4a37-b4e7-64870420314c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.882266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.882855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.883129 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.883464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.884053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.885942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:07.912593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1134/2290] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:07.916687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741821-bd8d-44b1-a8f4-7a9558122889 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:07.926963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741821-bd8d-44b1-a8f4-7a9558122889 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:07.982271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:07.982898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:07.983172 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:07.983468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:07.984092 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:07.986074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.005975 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1157/2291] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:07 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:08.009901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.020747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac83d9-88b4-4171-9c3b-d5c8843f50b3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.025350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.025961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.026221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.026488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.027117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.029356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.039676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1135/2292] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:08.043612 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.053208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bbf5d-247e-4e6f-92c6-34c2575b3b7c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.057733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.058353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.058627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.058894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] PATH_INFO: `/v3/projects/a983c8d5795b4eb2acaf2e1a7c3d8789/users/4dc0e3a97a4e4ed19c31077be56c982d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.073695 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a983c8d5795b4eb2acaf2e1a7c3d8789, user_id=4dc0e3a97a4e4ed19c31077be56c982d, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.077226 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.089572 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1158/2293] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:08 2022] PUT /identity/v3/projects/a983c8d5795b4eb2acaf2e1a7c3d8789/users/4dc0e3a97a4e4ed19c31077be56c982d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:08.093511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.103353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841c993-75c5-435f-bae7-d6b439931476 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.152655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:08.162945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f39266-fe8c-4d5c-9a78-98a62c200749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.163582 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39266-fe8c-4d5c-9a78-98a62c200749 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.163905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39266-fe8c-4d5c-9a78-98a62c200749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.164134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39266-fe8c-4d5c-9a78-98a62c200749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.165159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f39266-fe8c-4d5c-9a78-98a62c200749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.167404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f39266-fe8c-4d5c-9a78-98a62c200749 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.178973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1136/2294] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:08.183621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.195234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e35d49-8dd1-4805-923b-289ea742d284 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.200468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.201561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.201832 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.202099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] PATH_INFO: `/v3/projects/a983c8d5795b4eb2acaf2e1a7c3d8789/users/4dc0e3a97a4e4ed19c31077be56c982d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.204027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a983c8d5795b4eb2acaf2e1a7c3d8789, user_id=4dc0e3a97a4e4ed19c31077be56c982d, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.206644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.216729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1159/2295] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:08 2022] PUT /identity/v3/projects/a983c8d5795b4eb2acaf2e1a7c3d8789/users/4dc0e3a97a4e4ed19c31077be56c982d/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:08.227668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc257c-8c84-455d-9282-1a61a1679cd8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.227959 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc257c-8c84-455d-9282-1a61a1679cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.228220 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc257c-8c84-455d-9282-1a61a1679cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.255565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d5dc257c-8c84-455d-9282-1a61a1679cd8 None None] MFA Rules not processed for user `4dc0e3a97a4e4ed19c31077be56c982d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:08.256790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d5dc257c-8c84-455d-9282-1a61a1679cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:08.322334 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1137/2296] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:08.335393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.337086 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:08.398954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a5d5-c3cc-4c35-8d51-5408edb87de5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.402621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.403230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.403520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.403792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.412672 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.414105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.424914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1160/2297] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:08.727423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc257c-8c84-455d-9282-1a61a1679cd8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.737160 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc257c-8c84-455d-9282-1a61a1679cd8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.741712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dc0e3a97a4e4ed19c31077be56c982d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a983c8d5795b4eb2acaf2e1a7c3d8789', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.742305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.742586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.742861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.743536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:08.743878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.745473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.757516 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1138/2298] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:08.761620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.770626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde4163-028c-44e4-98cb-95481e2f1a8f service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.775124 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dc0e3a97a4e4ed19c31077be56c982d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a983c8d5795b4eb2acaf2e1a7c3d8789', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.775743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.776024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.776297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.776923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.779335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.805558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1161/2299] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:08 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:08.809531 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.819376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efa70c-093d-4bfe-98b7-d18d29807901 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.877934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dc0e3a97a4e4ed19c31077be56c982d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a983c8d5795b4eb2acaf2e1a7c3d8789', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.878537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.878817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.879094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.879768 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:08.880070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.881679 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.894463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1139/2300] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:08.934756 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:08.945494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda6556-d1e5-429c-a36f-c7c872867134 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:08.952090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dc0e3a97a4e4ed19c31077be56c982d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a983c8d5795b4eb2acaf2e1a7c3d8789', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:08.952717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:08.953002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:08.953277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] PATH_INFO: `/v3/projects/822cc9e5f6f44274981efb757cfcb71e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:08.954412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorizing `identity:delete_project(project_id=822cc9e5f6f44274981efb757cfcb71e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:08.956370 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:08.983958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:09.007954 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822cc9e5f6f44274981efb757cfcb71e', 'request_id': 'req-9990b62b-7b99-4af4-bcc9-53f08ea764d4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.013430 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822cc9e5f6f44274981efb757cfcb71e', 'request_id': 'req-9990b62b-7b99-4af4-bcc9-53f08ea764d4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.020157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822cc9e5f6f44274981efb757cfcb71e', 'request_id': 'req-9990b62b-7b99-4af4-bcc9-53f08ea764d4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.034982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] The token cache is being invalidate because project 822cc9e5f6f44274981efb757cfcb71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:09.035289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.035913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9990b62b-7b99-4af4-bcc9-53f08ea764d4 tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.037389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1162/2301] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:08 2022] DELETE /identity/v3/projects/822cc9e5f6f44274981efb757cfcb71e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:09.270540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:09.272277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:09.331931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:09.338037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f129697-4a4a-44a3-8580-a2c57702c7dc tempest-VolumeQuotaClassesTest-274795419 tempest-VolumeQuotaClassesTest-274795419-project-admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:09.394697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:09.395348 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:09.395629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:09.395901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] PATH_INFO: `/v3/users/940ad712d03947a6b00fae12944f78ff` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:09.397015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=940ad712d03947a6b00fae12944f78ff)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:09.398952 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:09.444219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '940ad712d03947a6b00fae12944f78ff'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.444836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '940ad712d03947a6b00fae12944f78ff'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.451904 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '940ad712d03947a6b00fae12944f78ff'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.460375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '940ad712d03947a6b00fae12944f78ff'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.460992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '940ad712d03947a6b00fae12944f78ff'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.477967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1140/2302] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:09 2022] DELETE /identity/v3/users/940ad712d03947a6b00fae12944f78ff => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:09.482926 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:09.483287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:09.483531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:09.502739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:09.503963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:09.561366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1163/2303] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:09.570915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:09.572328 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:09.628574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cb340-2caa-485d-8d3f-96e7a53d4d14 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:09.632186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:09.632774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:09.633049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:09.633319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:09.642564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:09.644024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:09.653193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1141/2304] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:09.921431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:09.922866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:09.935628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4c525-d5a7-4282-a0d0-e397a95e2191 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:09.940130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:09.940745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:09.941018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:09.941283 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] PATH_INFO: `/v3/projects/3c1a6fa0c17348e6a8aea03c0e787d2a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:09.942487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1a6fa0c17348e6a8aea03c0e787d2a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:09.944588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:09.991012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1a6fa0c17348e6a8aea03c0e787d2a', 'request_id': 'req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:09.996545 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1a6fa0c17348e6a8aea03c0e787d2a', 'request_id': 'req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.003165 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1a6fa0c17348e6a8aea03c0e787d2a', 'request_id': 'req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.014674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] The token cache is being invalidate because project 3c1a6fa0c17348e6a8aea03c0e787d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:10.014953 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.015571 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.017334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1164/2305] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:09 2022] DELETE /identity/v3/projects/3c1a6fa0c17348e6a8aea03c0e787d2a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:10.020948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.030613 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68869ac-522c-4763-b152-3c5cb84afb7e service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:10.050150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:10.089461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:10.090084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.090363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.090635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] PATH_INFO: `/v3/users/4dc0e3a97a4e4ed19c31077be56c982d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.091766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc0e3a97a4e4ed19c31077be56c982d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:10.093690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:10.138833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc0e3a97a4e4ed19c31077be56c982d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.139419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc0e3a97a4e4ed19c31077be56c982d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.146136 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc0e3a97a4e4ed19c31077be56c982d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.153218 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc0e3a97a4e4ed19c31077be56c982d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.153898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e85893dd-f9d8-4964-af24-356159eba6d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc0e3a97a4e4ed19c31077be56c982d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.173007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1142/2306] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:10 2022] DELETE /identity/v3/users/4dc0e3a97a4e4ed19c31077be56c982d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:10.454419 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.455868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:10.469507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c9023-8ebc-4422-bdf2-05a40a66b35b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:10.503816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:10.529773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:10.530392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.530673 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.530943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] PATH_INFO: `/v3/projects/a983c8d5795b4eb2acaf2e1a7c3d8789` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.532080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a983c8d5795b4eb2acaf2e1a7c3d8789)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:10.533980 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:10.589818 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a983c8d5795b4eb2acaf2e1a7c3d8789', 'request_id': 'req-070177d6-2ea2-4be7-a3eb-2c82f7492924'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.595768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a983c8d5795b4eb2acaf2e1a7c3d8789', 'request_id': 'req-070177d6-2ea2-4be7-a3eb-2c82f7492924'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.602059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a983c8d5795b4eb2acaf2e1a7c3d8789', 'request_id': 'req-070177d6-2ea2-4be7-a3eb-2c82f7492924'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.615291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] The token cache is being invalidate because project a983c8d5795b4eb2acaf2e1a7c3d8789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:10.615575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.616185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-070177d6-2ea2-4be7-a3eb-2c82f7492924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:10.617646 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1165/2307] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:10 2022] DELETE /identity/v3/projects/a983c8d5795b4eb2acaf2e1a7c3d8789 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:10.622550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56fac-6a8d-413a-aca5-cc76ad056492 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.623047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56fac-6a8d-413a-aca5-cc76ad056492 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.623157 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56fac-6a8d-413a-aca5-cc76ad056492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.642221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e7c56fac-6a8d-413a-aca5-cc76ad056492 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:10.643437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e7c56fac-6a8d-413a-aca5-cc76ad056492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:10.701343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1143/2308] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:10.708943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f75fb-121c-463e-9011-e5def9dd468f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.709251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f75fb-121c-463e-9011-e5def9dd468f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.709528 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f75fb-121c-463e-9011-e5def9dd468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.730275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a63f75fb-121c-463e-9011-e5def9dd468f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:10.731514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a63f75fb-121c-463e-9011-e5def9dd468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:10.739144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1166/2309] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:10.743188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c56fac-6a8d-413a-aca5-cc76ad056492 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.752614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c56fac-6a8d-413a-aca5-cc76ad056492 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:10.757080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:10.757659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.757931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.758197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.758824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:10.759119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:10.760733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:10.772530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1144/2310] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:10.776501 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f75fb-121c-463e-9011-e5def9dd468f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.786060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f75fb-121c-463e-9011-e5def9dd468f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:10.790725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:10.791347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.791622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.791891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.792512 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:10.794429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:10.820260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1167/2311] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:10 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:10.824558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.834827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d508671-3831-4e04-a010-0676bdd1433d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:10.863072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:10.893340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:10.893940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.894215 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.894483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.895109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:10.897061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:10.917753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1145/2312] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:10 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:10.921877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.932200 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386748b-8553-4794-9a98-2b964582991c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:10.936781 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:10.937367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.937642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.937908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.938536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:10.940149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:10.951110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1168/2313] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:10.955098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.964845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a6a26-6da6-43b6-bee3-53e70e355430 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:10.969997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee62b3-61cf-400e-a873-0974a68f6bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:10.970575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee62b3-61cf-400e-a873-0974a68f6bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:10.970848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee62b3-61cf-400e-a873-0974a68f6bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:10.971114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee62b3-61cf-400e-a873-0974a68f6bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:10.971872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee62b3-61cf-400e-a873-0974a68f6bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:10.973454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee62b3-61cf-400e-a873-0974a68f6bc9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:10.983066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1146/2314] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:10.986807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:10.996610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874cad0-d54f-4688-9057-40bfa5c67bb6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.001140 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.001771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.002046 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.002313 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] PATH_INFO: `/v3/projects/37054d2e035c43d59bc7a87b167a5b08/users/12e83aadd234484495ea142d4ae2d6ab/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.016879 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=37054d2e035c43d59bc7a87b167a5b08, user_id=12e83aadd234484495ea142d4ae2d6ab, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.019542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.032118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1169/2315] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:10 2022] PUT /identity/v3/projects/37054d2e035c43d59bc7a87b167a5b08/users/12e83aadd234484495ea142d4ae2d6ab/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:11.042271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb080-4dea-48ea-9911-bb4f47417ba0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.042548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb080-4dea-48ea-9911-bb4f47417ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.042816 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb080-4dea-48ea-9911-bb4f47417ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.071269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-05deb080-4dea-48ea-9911-bb4f47417ba0 None None] MFA Rules not processed for user `12e83aadd234484495ea142d4ae2d6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:11.072490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-05deb080-4dea-48ea-9911-bb4f47417ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:11.135473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1147/2316] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:11.144915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.217686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b85e1-0f66-42d6-8d17-f4da983a64ad admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.222656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.223299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.223580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.223849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.224467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.226468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.232786 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:11.263296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1170/2317] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:11 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:11.265324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deb080-4dea-48ea-9911-bb4f47417ba0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.276373 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deb080-4dea-48ea-9911-bb4f47417ba0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.341629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.342797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.343066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.343362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.343985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.345879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.366772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1148/2318] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:11 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:11.371081 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.382157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4cebb-de7a-4230-a3e5-3f7ffc453c4b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.387312 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.387970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.388345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.389116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.389855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.391607 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.401963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1171/2319] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:11.406796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.417395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbc6b3-966b-41bc-9bfe-68819c2b48b4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.422005 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.422861 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.423139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.423427 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] PATH_INFO: `/v3/projects/7f64c721bd5d497983b196c9aa60d766/users/ca9c21edc8414477949886edcb6100af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.440710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f64c721bd5d497983b196c9aa60d766, user_id=ca9c21edc8414477949886edcb6100af, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.443483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.457280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1149/2320] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:11 2022] PUT /identity/v3/projects/7f64c721bd5d497983b196c9aa60d766/users/ca9c21edc8414477949886edcb6100af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:11.461319 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.471558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ce56a-3ebc-4e12-84a2-75d2d01edd3d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.531339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf65323-b2a6-4783-9696-abe3b9fe9f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.532549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf65323-b2a6-4783-9696-abe3b9fe9f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.532829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf65323-b2a6-4783-9696-abe3b9fe9f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.533101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf65323-b2a6-4783-9696-abe3b9fe9f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.533746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf65323-b2a6-4783-9696-abe3b9fe9f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.535398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf65323-b2a6-4783-9696-abe3b9fe9f13 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.545220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1172/2321] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:11.549346 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.556579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:11.561223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b1e19-0e4e-40a8-b94e-962b2b7d9094 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.567080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.567762 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.568034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.568301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] PATH_INFO: `/v3/projects/7f64c721bd5d497983b196c9aa60d766/users/ca9c21edc8414477949886edcb6100af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.570180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f64c721bd5d497983b196c9aa60d766, user_id=ca9c21edc8414477949886edcb6100af, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.572887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.588177 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1150/2322] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:11 2022] PUT /identity/v3/projects/7f64c721bd5d497983b196c9aa60d766/users/ca9c21edc8414477949886edcb6100af/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:11.595390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.595701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.595978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.623562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] MFA Rules not processed for user `ca9c21edc8414477949886edcb6100af`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:11.624768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:11.691701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1173/2323] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:11.703329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.705101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:11.761207 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011886c-2582-43b5-802b-d2f0050e20ae admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.765359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.765945 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.766224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.766499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.775255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.776684 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.785892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1151/2324] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:11 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:11.832224 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:11.833593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:11.845234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cb437-b412-40b2-9573-86cd4951a33b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:11.848769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:11.849349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:11.849625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:11.849897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:11.902016 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:11.904375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:11.906567 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:11.916189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1174/2325] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:13.633307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:13.635103 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:13.739660 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c79c27-6209-4023-9d2f-4f1c7560e2e4 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:13.745020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4087351-da10-4575-8d6e-38f611a78070 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:13.745708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4087351-da10-4575-8d6e-38f611a78070 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:13.746056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4087351-da10-4575-8d6e-38f611a78070 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:13.746397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4087351-da10-4575-8d6e-38f611a78070 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:13.748872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4087351-da10-4575-8d6e-38f611a78070 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:13.761712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4087351-da10-4575-8d6e-38f611a78070 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:13.763487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4087351-da10-4575-8d6e-38f611a78070 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:13.775337 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1152/2326] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:14.812830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:14.814896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:14.912342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed627b0e-a0f7-45ee-b109-aef3477d9b0e service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:14.918486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:14.919130 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:14.919439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:14.919759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] PATH_INFO: `/v3/users/99a16caa037b4cf6b1e62a5779e04e56` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:14.920933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a16caa037b4cf6b1e62a5779e04e56)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:14.922859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:14.976069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a16caa037b4cf6b1e62a5779e04e56'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:14.976706 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a16caa037b4cf6b1e62a5779e04e56'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:14.984210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a16caa037b4cf6b1e62a5779e04e56'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:14.984918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a16caa037b4cf6b1e62a5779e04e56'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:14.997300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a16caa037b4cf6b1e62a5779e04e56'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.010649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1175/2327] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:14 2022] DELETE /identity/v3/users/99a16caa037b4cf6b1e62a5779e04e56 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:15.016130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.016629 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.016756 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.036143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:15.037340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.096970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1153/2328] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:15.107098 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:15.108549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.189880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f7cc3-17c0-422a-9c36-e622f43667f8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:15.193952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:15.194801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.195329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.195777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.211871 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:15.213322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:15.226795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1176/2329] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:15.484797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:15.488184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.499327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:15.507638 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee632741-a393-47d0-a95f-c9932122be80 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:15.519122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:15.520101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.520541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.520978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] PATH_INFO: `/v3/users/12e83aadd234484495ea142d4ae2d6ab` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.522302 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e83aadd234484495ea142d4ae2d6ab)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:15.524467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:15.594188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e83aadd234484495ea142d4ae2d6ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.595528 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e83aadd234484495ea142d4ae2d6ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.606973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e83aadd234484495ea142d4ae2d6ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.618635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e83aadd234484495ea142d4ae2d6ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.620063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4999b3b8-7e84-4f9b-a15a-f58768de66a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e83aadd234484495ea142d4ae2d6ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.624964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:15.626381 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.653006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1154/2330] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:15 2022] DELETE /identity/v3/users/12e83aadd234484495ea142d4ae2d6ab => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:15.653248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f166d3-7f3e-4c4e-819d-e664e851c4c8 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:15.655415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.655755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.656030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.670309 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:15.678579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:15.679956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.730369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:15.731116 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.731497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.731848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] PATH_INFO: `/v3/projects/6e2a9c2575114667bd8ed552b3725ed7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.733798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2a9c2575114667bd8ed552b3725ed7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:15.735792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:15.753269 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1155/2331] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:15.762833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:15.764427 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.797164 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2a9c2575114667bd8ed552b3725ed7', 'request_id': 'req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.807787 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2a9c2575114667bd8ed552b3725ed7', 'request_id': 'req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.815936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2a9c2575114667bd8ed552b3725ed7', 'request_id': 'req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.831225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] The token cache is being invalidate because project 6e2a9c2575114667bd8ed552b3725ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:15.831435 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abcc09-0268-4715-bae9-7d34a75a2d3a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:15.831574 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.832310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e3d0cd18-c8ac-4a66-b6dd-5cb814094c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:15.834145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1177/2332] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:15 2022] DELETE /identity/v3/projects/6e2a9c2575114667bd8ed552b3725ed7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:15.838761 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946efa3a-1368-46ee-9d7a-820130b84752 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.839020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946efa3a-1368-46ee-9d7a-820130b84752 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.839334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946efa3a-1368-46ee-9d7a-820130b84752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.862891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-946efa3a-1368-46ee-9d7a-820130b84752 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:15.864596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:15.865005 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-946efa3a-1368-46ee-9d7a-820130b84752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.900765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:15.900765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.900765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.901266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.906330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.946077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1178/2333] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:15.946750 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dff0e-bff1-417f-9c1d-61e8dcaa3460 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.947060 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dff0e-bff1-417f-9c1d-61e8dcaa3460 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.947352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dff0e-bff1-417f-9c1d-61e8dcaa3460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.967097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-257dff0e-bff1-417f-9c1d-61e8dcaa3460 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:15.968310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-257dff0e-bff1-417f-9c1d-61e8dcaa3460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:15.974178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:15.975653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:15.976012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1179/2334] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:15.980021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dff0e-bff1-417f-9c1d-61e8dcaa3460 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:15.986030 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1156/2335] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:15.990680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dff0e-bff1-417f-9c1d-61e8dcaa3460 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:15.996048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:15.996647 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:15.996994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:15.997336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:15.998285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:15.998596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.000432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.012902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1180/2336] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:16.017375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.027504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e42da7-dc65-4044-ab46-e5c67d21872d service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.032855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.033447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.033699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.033975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.034599 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.036535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.063457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1157/2337] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:16 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:16.067513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.077464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1da6-9dc4-4afc-a4ae-91379660771d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.105894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:16.142854 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.143494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.143770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.144036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.144657 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.146740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.171332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1181/2338] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:16 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:16.175120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.186515 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314323e-931d-4c1e-9a1c-2c956ff38a65 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.192309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.193091 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.193474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.193896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.194651 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.196403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.211294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1158/2339] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:16.213145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.223717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4aa80-291a-41ff-b9dc-50c322b5b99c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.228375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.229037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.229402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.229740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.230475 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.232723 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.244012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1182/2340] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:16.250135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.264716 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762252e-69d4-4fae-b8cd-76b2c550ded6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.265780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.267810 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:16.277614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6fa07-50b3-4657-8d8f-2333e51d0b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.278295 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6fa07-50b3-4657-8d8f-2333e51d0b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.278569 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6fa07-50b3-4657-8d8f-2333e51d0b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.278835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6fa07-50b3-4657-8d8f-2333e51d0b38 admin admin] PATH_INFO: `/v3/projects/8dd5c25527c5495596f40e63b3f31a69/users/4d33f09e872b44a281c4bc30dacf0c82/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.285785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e623e2-86a5-4f3a-ba94-acc28c3d9224 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.291487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.292113 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.292389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.292660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] PATH_INFO: `/v3/projects/37054d2e035c43d59bc7a87b167a5b08` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.294555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37054d2e035c43d59bc7a87b167a5b08)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.297480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.299093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6fa07-50b3-4657-8d8f-2333e51d0b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd5c25527c5495596f40e63b3f31a69, user_id=4d33f09e872b44a281c4bc30dacf0c82, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.302483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6fa07-50b3-4657-8d8f-2333e51d0b38 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.317357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1159/2341] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:16 2022] PUT /identity/v3/projects/8dd5c25527c5495596f40e63b3f31a69/users/4d33f09e872b44a281c4bc30dacf0c82/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:16.328501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.328786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.329057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.352107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37054d2e035c43d59bc7a87b167a5b08', 'request_id': 'req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.359928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] MFA Rules not processed for user `4d33f09e872b44a281c4bc30dacf0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:16.360107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37054d2e035c43d59bc7a87b167a5b08', 'request_id': 'req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.361134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:16.371702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37054d2e035c43d59bc7a87b167a5b08', 'request_id': 'req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.385996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] The token cache is being invalidate because project 37054d2e035c43d59bc7a87b167a5b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:16.385996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.386778 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.389472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1183/2342] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:16 2022] DELETE /identity/v3/projects/37054d2e035c43d59bc7a87b167a5b08 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:16.393532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.395128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:16.444288 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1160/2343] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:16.457040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.458790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:16.475154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0291eb6-9b80-47cd-9ea7-9e38f2fad423 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.477447 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:16.480097 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.480725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.481006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.481291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] PATH_INFO: `/v3/users/ca9c21edc8414477949886edcb6100af` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.482627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9c21edc8414477949886edcb6100af)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.484818 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.519481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:16.534771 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9c21edc8414477949886edcb6100af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.535677 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9c21edc8414477949886edcb6100af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.538057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d351f-96fe-4cee-a3bb-c76bbe123395 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.542339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9c21edc8414477949886edcb6100af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.542965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9c21edc8414477949886edcb6100af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.543640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.544247 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.544524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.544825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.546565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:16.555352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9300018e-393c-4853-a00c-3ef1fca1ec14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9c21edc8414477949886edcb6100af'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:16.557893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.559550 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:16.569369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1161/2344] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:16.570181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1184/2345] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:16 2022] DELETE /identity/v3/users/ca9c21edc8414477949886edcb6100af => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:16.861896 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:16.863917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:16.883846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c48e8-cf23-4983-bd4c-42d4aee3b87a service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:16.955843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:16.956478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:16.956755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:16.957024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] PATH_INFO: `/v3/projects/7f64c721bd5d497983b196c9aa60d766` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:16.958286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f64c721bd5d497983b196c9aa60d766)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:16.960219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.012162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f64c721bd5d497983b196c9aa60d766', 'request_id': 'req-544c3e19-c165-4115-bccd-1f7cf1a6f72b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:17.018148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f64c721bd5d497983b196c9aa60d766', 'request_id': 'req-544c3e19-c165-4115-bccd-1f7cf1a6f72b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:17.025264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f64c721bd5d497983b196c9aa60d766', 'request_id': 'req-544c3e19-c165-4115-bccd-1f7cf1a6f72b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:17.036793 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] The token cache is being invalidate because project 7f64c721bd5d497983b196c9aa60d766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:17.037076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:17.037719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-544c3e19-c165-4115-bccd-1f7cf1a6f72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:17.039229 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1162/2346] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:16 2022] DELETE /identity/v3/projects/7f64c721bd5d497983b196c9aa60d766 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:17.044034 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4c94-f1a2-424e-8495-ff1bc4a057a4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.044477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4c94-f1a2-424e-8495-ff1bc4a057a4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.045259 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4c94-f1a2-424e-8495-ff1bc4a057a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.064892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad0c4c94-f1a2-424e-8495-ff1bc4a057a4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:17.066084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad0c4c94-f1a2-424e-8495-ff1bc4a057a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:17.124349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1185/2347] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:17.132048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6697-1bc4-43ee-8c57-56634dc2aa7f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.132325 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6697-1bc4-43ee-8c57-56634dc2aa7f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.132593 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6697-1bc4-43ee-8c57-56634dc2aa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.152252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cbcf6697-1bc4-43ee-8c57-56634dc2aa7f None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:17.153437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbcf6697-1bc4-43ee-8c57-56634dc2aa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:17.161323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1163/2348] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:17.165417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4c94-f1a2-424e-8495-ff1bc4a057a4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:17.175513 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4c94-f1a2-424e-8495-ff1bc4a057a4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:17.180070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:17.180653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.180936 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.181206 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.181842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:17.182139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:17.183758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.195532 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1186/2349] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:17.199543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf6697-1bc4-43ee-8c57-56634dc2aa7f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:17.208799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf6697-1bc4-43ee-8c57-56634dc2aa7f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:17.213395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:17.213991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.214248 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.214561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.215300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:17.217445 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.243654 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1164/2350] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:17 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:17.248472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:17.258624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb77d0d-61ea-44a7-9c3c-ed76f646453d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:17.315725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:17.316326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.316598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.316860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.317496 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:17.319439 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.339932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1187/2351] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:17 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:17.344202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:17.349539 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:17.355646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5890b8-6ad1-418e-9548-78830d30ed9d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:17.361757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:17.362375 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.362676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.362946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.363614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:17.365250 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.375746 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1165/2352] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:17.379505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:17.389102 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205aacc-ca27-47ad-86b2-fe56eccc4321 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:17.393651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b989135-784f-4381-8f15-f35bd73e75ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:17.394230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989135-784f-4381-8f15-f35bd73e75ed admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.394627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989135-784f-4381-8f15-f35bd73e75ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.394825 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989135-784f-4381-8f15-f35bd73e75ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.395638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b989135-784f-4381-8f15-f35bd73e75ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:17.397246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b989135-784f-4381-8f15-f35bd73e75ed admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.408276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1188/2353] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:17.412433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:17.423548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48f1f0-4daa-4a53-a936-958a353263a7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:17.428262 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:17.428931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.429209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.429480 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] PATH_INFO: `/v3/projects/c5fc51cd319448d5a83b6038c4610d61/users/e6a48b2767514860bc5d7481f7e3ab87/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.443938 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fc51cd319448d5a83b6038c4610d61, user_id=e6a48b2767514860bc5d7481f7e3ab87, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:17.446586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.459192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1166/2354] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:17 2022] PUT /identity/v3/projects/c5fc51cd319448d5a83b6038c4610d61/users/e6a48b2767514860bc5d7481f7e3ab87/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:17.464021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04115b-095f-448d-a270-1fbd4ceae0a8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.464304 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04115b-095f-448d-a270-1fbd4ceae0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.464560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04115b-095f-448d-a270-1fbd4ceae0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.485806 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ef04115b-095f-448d-a270-1fbd4ceae0a8 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:17.487011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ef04115b-095f-448d-a270-1fbd4ceae0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:17.500203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:17.550702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1189/2355] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:17.560268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:17.562051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:17.621655 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ca16-f54f-4ba5-8dc4-0350cd0fea0d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:17.625385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:17.625991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:17.626264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:17.626529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:17.636490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:17.637829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:17.648386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1167/2356] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:18.466394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad050c66-b1c5-4bb8-873e-30f46a2d5938 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:18.466931 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad050c66-b1c5-4bb8-873e-30f46a2d5938 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:18.467053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad050c66-b1c5-4bb8-873e-30f46a2d5938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:18.494411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad050c66-b1c5-4bb8-873e-30f46a2d5938 None None] MFA Rules not processed for user `e6a48b2767514860bc5d7481f7e3ab87`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:18.496087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad050c66-b1c5-4bb8-873e-30f46a2d5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:18.569568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1190/2357] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:18.580450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:18.592414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed4da2-3c55-4ab6-9511-48af5c0745f0 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:18.597416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:18.598015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:18.598294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:18.598565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:18.599260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:18.601275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:18.631449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1168/2358] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:18 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:18.636230 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad050c66-b1c5-4bb8-873e-30f46a2d5938 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:18.647558 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad050c66-b1c5-4bb8-873e-30f46a2d5938 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:18.707775 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:18.708426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:18.708708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:18.708979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:18.709602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:18.711562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:18.732838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1191/2359] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:18 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:18.736832 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:18.747047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24653d03-d172-4553-a380-a3bf55ae6a03 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:18.752254 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:18.752842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:18.753119 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:18.753387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:18.754010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:18.755620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:18.767688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1169/2360] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:18.772001 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:18.781770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99479ad4-6780-47dd-9ee0-494f91d41992 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:18.786401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:18.787039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:18.787349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:18.787619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] PATH_INFO: `/v3/projects/5096d5eb5cc341ccb47850ae73ef8e6e/users/714daf22c961443391cb43bedd66b3d2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:18.802347 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5096d5eb5cc341ccb47850ae73ef8e6e, user_id=714daf22c961443391cb43bedd66b3d2, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:18.805276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:18.818174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1192/2361] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:18 2022] PUT /identity/v3/projects/5096d5eb5cc341ccb47850ae73ef8e6e/users/714daf22c961443391cb43bedd66b3d2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:18.822060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:18.832280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bebb59-e8ee-40f5-9952-5c169316119c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:18.888547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:18.889139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:18.889411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:18.889678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:18.890300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:18.892073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:18.901588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1170/2362] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:18.906248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:18.916282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb431a-ac1b-4d43-81a8-618cd4ab8b21 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:18.921425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c32e7a-949a-46a8-9c03-ee0806fe3817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:18.922053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c32e7a-949a-46a8-9c03-ee0806fe3817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:18.922324 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c32e7a-949a-46a8-9c03-ee0806fe3817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:18.922591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c32e7a-949a-46a8-9c03-ee0806fe3817 admin admin] PATH_INFO: `/v3/projects/5096d5eb5cc341ccb47850ae73ef8e6e/users/714daf22c961443391cb43bedd66b3d2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:18.924464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c32e7a-949a-46a8-9c03-ee0806fe3817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5096d5eb5cc341ccb47850ae73ef8e6e, user_id=714daf22c961443391cb43bedd66b3d2, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:18.927072 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c32e7a-949a-46a8-9c03-ee0806fe3817 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:18.932409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:18.938628 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1193/2363] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:18 2022] PUT /identity/v3/projects/5096d5eb5cc341ccb47850ae73ef8e6e/users/714daf22c961443391cb43bedd66b3d2/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:19.316728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:19.318464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:19.336231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74f135-2ac4-4139-a69e-05364f04003e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:19.390767 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:19.428330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:19.428970 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:19.429243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:19.429512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] PATH_INFO: `/v3/users/4d33f09e872b44a281c4bc30dacf0c82` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:19.430795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d33f09e872b44a281c4bc30dacf0c82)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:19.433179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:19.500903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d33f09e872b44a281c4bc30dacf0c82'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:19.501711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d33f09e872b44a281c4bc30dacf0c82'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:19.509548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d33f09e872b44a281c4bc30dacf0c82'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:19.520206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d33f09e872b44a281c4bc30dacf0c82'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:19.520838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d33f09e872b44a281c4bc30dacf0c82'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:19.542574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1171/2364] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:19 2022] DELETE /identity/v3/users/4d33f09e872b44a281c4bc30dacf0c82 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:19.546970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46fe38-e9f0-4172-aa7d-39c92397b0f3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:19.547782 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46fe38-e9f0-4172-aa7d-39c92397b0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:19.548207 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46fe38-e9f0-4172-aa7d-39c92397b0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:19.572314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4e46fe38-e9f0-4172-aa7d-39c92397b0f3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:19.573746 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4e46fe38-e9f0-4172-aa7d-39c92397b0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:19.659583 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1194/2365] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:19.674227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:19.675736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:19.707401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:19.707737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:19.707951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:19.741425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] MFA Rules not processed for user `714daf22c961443391cb43bedd66b3d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:19.742848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:19.773275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be75f8-f191-420d-a5fc-07918eba8fdb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:19.777197 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:19.777903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:19.778282 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:19.778740 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:19.789253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:19.790792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:19.801279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1172/2366] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:19.820937 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1195/2367] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:19.835350 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:19.836995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:19.888205 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:19.908412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6c07-696a-43e6-9fee-a4797e5a3a67 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:19.912614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f38a5b-19c0-4806-9e08-c95d4f5b2c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:19.913256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f38a5b-19c0-4806-9e08-c95d4f5b2c32 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:19.913530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f38a5b-19c0-4806-9e08-c95d4f5b2c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:19.914324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f38a5b-19c0-4806-9e08-c95d4f5b2c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:19.915769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e4f38a5b-19c0-4806-9e08-c95d4f5b2c32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:19.982433 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f38a5b-19c0-4806-9e08-c95d4f5b2c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:19.984556 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f38a5b-19c0-4806-9e08-c95d4f5b2c32 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:19.993701 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1173/2368] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:20.097656 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.099468 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:20.119891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb5d9-9450-47d8-a330-489b49d27ecd None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.124536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.125261 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.125610 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.125952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] PATH_INFO: `/v3/projects/8dd5c25527c5495596f40e63b3f31a69` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.127150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd5c25527c5495596f40e63b3f31a69)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.129433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.199977 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd5c25527c5495596f40e63b3f31a69', 'request_id': 'req-27ba0cda-d296-4ad7-a85c-11aff53b1980'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:20.207115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd5c25527c5495596f40e63b3f31a69', 'request_id': 'req-27ba0cda-d296-4ad7-a85c-11aff53b1980'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:20.214174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd5c25527c5495596f40e63b3f31a69', 'request_id': 'req-27ba0cda-d296-4ad7-a85c-11aff53b1980'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:20.226629 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] The token cache is being invalidate because project 8dd5c25527c5495596f40e63b3f31a69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:20.226915 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:20.227551 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-27ba0cda-d296-4ad7-a85c-11aff53b1980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:20.229048 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1196/2369] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:20 2022] DELETE /identity/v3/projects/8dd5c25527c5495596f40e63b3f31a69 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:20.234052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102fc65-1ed9-4f29-a52c-5257acc6a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.234383 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102fc65-1ed9-4f29-a52c-5257acc6a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.234661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102fc65-1ed9-4f29-a52c-5257acc6a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.258906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2102fc65-1ed9-4f29-a52c-5257acc6a1c1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:20.260211 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2102fc65-1ed9-4f29-a52c-5257acc6a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:20.331185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1174/2370] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:20.340246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73bf26-1c20-46c7-8229-b557fffc58af None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.340625 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73bf26-1c20-46c7-8229-b557fffc58af None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.340901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73bf26-1c20-46c7-8229-b557fffc58af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.357794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:20.364236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8a73bf26-1c20-46c7-8229-b557fffc58af None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:20.368970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a73bf26-1c20-46c7-8229-b557fffc58af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:20.378326 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1197/2371] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:20.382675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102fc65-1ed9-4f29-a52c-5257acc6a1c1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.392985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102fc65-1ed9-4f29-a52c-5257acc6a1c1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.397581 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.398176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.398452 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.398780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.399705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:20.400040 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.401819 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.413796 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1175/2372] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:20.418131 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73bf26-1c20-46c7-8229-b557fffc58af None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.427390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73bf26-1c20-46c7-8229-b557fffc58af None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.431912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.432605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.432887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.433154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.433777 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.436217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.464161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1198/2373] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:20 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:20.468318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.479142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa457ef2-0b70-4e99-92b4-219d33e41f48 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.533479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.533997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.534271 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.534544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.535152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.537078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.561469 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1176/2374] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:20 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:20.562660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.574662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e7789-803c-4937-b28e-492967b30e73 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.580519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.581186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.581531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.581885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.582590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.584290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.594269 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1199/2375] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:20.598195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.608228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c929716-a045-4481-b02b-8162e40b962b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.614315 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9323b8-5265-43ad-af42-2c866256162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.614906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9323b8-5265-43ad-af42-2c866256162c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.615184 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9323b8-5265-43ad-af42-2c866256162c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.615485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9323b8-5265-43ad-af42-2c866256162c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.616113 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9323b8-5265-43ad-af42-2c866256162c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.617708 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9323b8-5265-43ad-af42-2c866256162c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.618946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:20.630950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1177/2376] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:20.634976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.646650 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f5af6-4130-4475-ab62-8218ddb16d49 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.651460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.652108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.652415 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.652687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] PATH_INFO: `/v3/projects/d28e4df6afda44cab21f12f0c2303b9a/users/ec616de6c92f41658e9003ba4ff9c875/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.668477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28e4df6afda44cab21f12f0c2303b9a, user_id=ec616de6c92f41658e9003ba4ff9c875, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.671143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.684406 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1200/2377] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:20 2022] PUT /identity/v3/projects/d28e4df6afda44cab21f12f0c2303b9a/users/ec616de6c92f41658e9003ba4ff9c875/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:20.695533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.695981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.696066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.725035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] MFA Rules not processed for user `ec616de6c92f41658e9003ba4ff9c875`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:20.726355 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:20.778736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.780520 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:20.799893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1178/2378] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:20.810643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:20.812671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:20.837853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63492fc2-b574-4353-a7a7-7da9a246f101 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.841654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.842251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.842642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.842914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.844768 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:20.869748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf083f-b7de-4a5c-8ca7-9949a74dbf76 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:20.873313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:20.874574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:20.874841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:20.875107 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:20.883631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.885053 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.894003 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1179/2379] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:20.909357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:20.910776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:20.921257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1201/2380] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:21.381494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:21.382888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:21.394098 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b781b4-687f-4505-84b7-472e9b31dc95 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:21.397724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:21.398311 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:21.398584 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:21.398851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:21.400541 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:21.465179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:21.466773 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:21.476984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1180/2381] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:24.266252 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:24.270996 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:24.344778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:24.346654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:24.347960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:24.372865 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbb626-3f99-40f7-831e-b362503fa31e service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:24.379626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:24.379857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072c300-06fd-458d-ab6c-e867db3df4af service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:24.380652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:24.381080 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:24.381507 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] PATH_INFO: `/v3/users/8d404b2974f64aa8a6c32adfc3f7b0bc` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:24.382983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d404b2974f64aa8a6c32adfc3f7b0bc)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:24.385711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:24.385970 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:24.386659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:24.386941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:24.387290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:24.390108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:24.408809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:24.410454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:24.418045 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:24.423909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1202/2382] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:24.478050 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d404b2974f64aa8a6c32adfc3f7b0bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.478711 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d404b2974f64aa8a6c32adfc3f7b0bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.485675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d404b2974f64aa8a6c32adfc3f7b0bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.494284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d404b2974f64aa8a6c32adfc3f7b0bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.494977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7df51fd-6999-446a-9759-28d40772d775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d404b2974f64aa8a6c32adfc3f7b0bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.515537 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1181/2383] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:24 2022] DELETE /identity/v3/users/8d404b2974f64aa8a6c32adfc3f7b0bc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:24.693866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:24.706014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639c0b3-37a5-4532-bf25-49574cfbf09f service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:24.764850 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:24.765471 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:24.765745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:24.766010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] PATH_INFO: `/v3/projects/721190b4f16142749dda7253941be2c3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:24.767120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=721190b4f16142749dda7253941be2c3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:24.769210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:24.816292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721190b4f16142749dda7253941be2c3', 'request_id': 'req-691fddb0-8581-42d0-9c0f-b79057dcca72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.821939 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721190b4f16142749dda7253941be2c3', 'request_id': 'req-691fddb0-8581-42d0-9c0f-b79057dcca72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.828890 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721190b4f16142749dda7253941be2c3', 'request_id': 'req-691fddb0-8581-42d0-9c0f-b79057dcca72'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.840920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] The token cache is being invalidate because project 721190b4f16142749dda7253941be2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:24.841208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.842453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-691fddb0-8581-42d0-9c0f-b79057dcca72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:24.844023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1203/2384] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:24 2022] DELETE /identity/v3/projects/721190b4f16142749dda7253941be2c3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:24.848860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ef7ee-77b4-46d3-a9d4-760f33c4cf0e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:24.849123 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ef7ee-77b4-46d3-a9d4-760f33c4cf0e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:24.849400 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ef7ee-77b4-46d3-a9d4-760f33c4cf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:24.869082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-780ef7ee-77b4-46d3-a9d4-760f33c4cf0e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:24.870277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-780ef7ee-77b4-46d3-a9d4-760f33c4cf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:24.928786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1182/2385] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:24.937790 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd6d8b-2659-49d6-b290-8400f1eb3a62 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:24.938080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd6d8b-2659-49d6-b290-8400f1eb3a62 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:24.938357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd6d8b-2659-49d6-b290-8400f1eb3a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:24.959772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fefd6d8b-2659-49d6-b290-8400f1eb3a62 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:24.961016 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fefd6d8b-2659-49d6-b290-8400f1eb3a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:24.969367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1204/2386] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:24.973630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ef7ee-77b4-46d3-a9d4-760f33c4cf0e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:24.984049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ef7ee-77b4-46d3-a9d4-760f33c4cf0e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:24.988635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:24.989227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:24.989504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:24.989776 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:24.990443 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:24.990748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:24.992405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.008135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1183/2387] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:25.009604 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd6d8b-2659-49d6-b290-8400f1eb3a62 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.018865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd6d8b-2659-49d6-b290-8400f1eb3a62 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.023664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.024540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.024827 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.025209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.025894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.027891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.057642 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1205/2388] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:25 2022] POST /identity/v3/projects => generated 353 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:25.059769 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.070210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d584475-df0c-4ea6-a172-a241378daa18 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.126471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.127108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.127415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.127687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.128305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.130213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.155674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1184/2389] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:25 2022] POST /identity/v3/users => generated 277 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:25.159602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.170023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef384e0d-21f7-4811-90ae-7ad661b78626 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.175328 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.175912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.176192 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.176458 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.177112 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.178763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.191322 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1206/2390] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:25.193274 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.203672 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee717b9f-dc5e-40b0-bb82-441df1d48ef1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.208468 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63ca97-23b2-4b41-b937-2a9c458283e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.209069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63ca97-23b2-4b41-b937-2a9c458283e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.209342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63ca97-23b2-4b41-b937-2a9c458283e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.209609 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63ca97-23b2-4b41-b937-2a9c458283e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.210234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63ca97-23b2-4b41-b937-2a9c458283e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.212000 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63ca97-23b2-4b41-b937-2a9c458283e1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.218089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:25.227525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1185/2391] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:25.229705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.240663 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55eb2-fdae-4893-8c9f-ddf3c4cc4869 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.245834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.246479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.246758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.247033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] PATH_INFO: `/v3/projects/03ce305ca2f54883947c3ec2c8c4d498/users/e267c52c0e5e4c09b92db4a85147033d/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.261842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ce305ca2f54883947c3ec2c8c4d498, user_id=e267c52c0e5e4c09b92db4a85147033d, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.265101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.277785 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1207/2392] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:25 2022] PUT /identity/v3/projects/03ce305ca2f54883947c3ec2c8c4d498/users/e267c52c0e5e4c09b92db4a85147033d/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:25.288930 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883a1f-01fa-44e4-9569-d76f80da368b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.289213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883a1f-01fa-44e4-9569-d76f80da368b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.289488 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883a1f-01fa-44e4-9569-d76f80da368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.317476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-83883a1f-01fa-44e4-9569-d76f80da368b None None] MFA Rules not processed for user `e267c52c0e5e4c09b92db4a85147033d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:25.318682 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-83883a1f-01fa-44e4-9569-d76f80da368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:25.384961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1186/2393] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:25.390368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.401987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:25.458773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ccd8f-8550-4fb0-aae5-c927a2eca768 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.463376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.464061 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.464401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.464734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.465411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.467979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.500070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1208/2394] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:25 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:25.501567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883a1f-01fa-44e4-9569-d76f80da368b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.515903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883a1f-01fa-44e4-9569-d76f80da368b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.585565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.586172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.586449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.586723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.587382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.589308 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.615358 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1187/2395] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:25 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:25.619578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.632302 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ac13b-803c-4da7-a9c4-56ecd0899f4a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.641447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.642578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.643178 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.643543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.644289 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.646171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.657988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1209/2396] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:25.662341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.673018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78a823-f8bc-408d-b40c-7c2f24f165f8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.679646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.680440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.680857 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.681315 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] PATH_INFO: `/v3/projects/9190b3ab69d04ff88a902d3cba24d1a2/users/3eb11773c4d84cc5a338c138995cea5f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.698410 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9190b3ab69d04ff88a902d3cba24d1a2, user_id=3eb11773c4d84cc5a338c138995cea5f, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.702226 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.717964 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1188/2397] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:25 2022] PUT /identity/v3/projects/9190b3ab69d04ff88a902d3cba24d1a2/users/3eb11773c4d84cc5a338c138995cea5f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:25.721866 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.740472 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14547a94-580f-4e6d-8ef8-123450d5f19b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.833748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2b58d-9b31-4e6c-9346-734e6f27933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.834292 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2b58d-9b31-4e6c-9346-734e6f27933a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.834619 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2b58d-9b31-4e6c-9346-734e6f27933a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.834892 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2b58d-9b31-4e6c-9346-734e6f27933a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.835557 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2b58d-9b31-4e6c-9346-734e6f27933a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.837154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2b58d-9b31-4e6c-9346-734e6f27933a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.850470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1210/2398] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:25.853426 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:25.863743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c85e6f-4eb5-49b1-8df4-51f371ec0e7f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:25.868830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:25.869255 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.869466 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.869725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] PATH_INFO: `/v3/projects/9190b3ab69d04ff88a902d3cba24d1a2/users/3eb11773c4d84cc5a338c138995cea5f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.872010 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9190b3ab69d04ff88a902d3cba24d1a2, user_id=3eb11773c4d84cc5a338c138995cea5f, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:25.875228 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:25.886526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1189/2399] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:25 2022] PUT /identity/v3/projects/9190b3ab69d04ff88a902d3cba24d1a2/users/3eb11773c4d84cc5a338c138995cea5f/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:25.898379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:25.898585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:25.898829 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:25.927361 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] MFA Rules not processed for user `3eb11773c4d84cc5a338c138995cea5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:25.928602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:26.001592 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1211/2400] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:25 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:26.012909 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:26.016913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:26.063055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:26.098850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbe701-a635-4bc8-9943-3d48d4462cbe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:26.102833 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:26.103532 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:26.103888 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:26.104213 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:26.114165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:26.115641 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:26.125921 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1190/2401] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:26.204114 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:26.205486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:26.215358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:26.219812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14766f05-ff31-4f74-b809-0abdb66f575e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:26.224260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e977d8-a026-4060-8471-d245e11e7fe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:26.224845 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e977d8-a026-4060-8471-d245e11e7fe6 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:26.225144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e977d8-a026-4060-8471-d245e11e7fe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:26.225426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e977d8-a026-4060-8471-d245e11e7fe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:26.281515 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e977d8-a026-4060-8471-d245e11e7fe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:26.282925 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e977d8-a026-4060-8471-d245e11e7fe6 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:26.291536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1212/2402] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:29.477642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:29.479063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:29.547175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40b8e2-a6f1-471b-8745-727d9e862432 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:29.551727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:29.552341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:29.552614 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:29.552885 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] PATH_INFO: `/v3/users/ec616de6c92f41658e9003ba4ff9c875` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:29.553986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec616de6c92f41658e9003ba4ff9c875)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:29.555939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:29.607841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec616de6c92f41658e9003ba4ff9c875'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:29.609205 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec616de6c92f41658e9003ba4ff9c875'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:29.616703 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec616de6c92f41658e9003ba4ff9c875'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:29.625369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec616de6c92f41658e9003ba4ff9c875'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:29.626002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec616de6c92f41658e9003ba4ff9c875'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:29.646628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1191/2403] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:29 2022] DELETE /identity/v3/users/ec616de6c92f41658e9003ba4ff9c875 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:29.650875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:29.651459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:29.651459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:29.673556 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:29.674793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:29.735688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1213/2404] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:29.745356 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:29.746845 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:29.793104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:29.805166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf16365-f319-45c4-8c3b-51a1c25e793a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:29.808974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd68c7-5524-4268-9d45-84ad82c0a65f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:29.809619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd68c7-5524-4268-9d45-84ad82c0a65f service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:29.809947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd68c7-5524-4268-9d45-84ad82c0a65f service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:29.810219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd68c7-5524-4268-9d45-84ad82c0a65f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:29.819490 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd68c7-5524-4268-9d45-84ad82c0a65f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:29.821411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd68c7-5524-4268-9d45-84ad82c0a65f service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:29.833505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1192/2405] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:30.104127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.105621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:30.131561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97310bf-0981-4650-8bb0-611d42b30429 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.136759 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.137373 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.137658 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.137927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] PATH_INFO: `/v3/projects/d28e4df6afda44cab21f12f0c2303b9a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.139021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28e4df6afda44cab21f12f0c2303b9a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.141865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.194424 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28e4df6afda44cab21f12f0c2303b9a', 'request_id': 'req-2d26b987-216e-40b0-ae10-199f6032e0ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:30.200834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28e4df6afda44cab21f12f0c2303b9a', 'request_id': 'req-2d26b987-216e-40b0-ae10-199f6032e0ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:30.207849 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28e4df6afda44cab21f12f0c2303b9a', 'request_id': 'req-2d26b987-216e-40b0-ae10-199f6032e0ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:30.220115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] The token cache is being invalidate because project d28e4df6afda44cab21f12f0c2303b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:30.220400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:30.221064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2d26b987-216e-40b0-ae10-199f6032e0ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:30.222568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1214/2406] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:30 2022] DELETE /identity/v3/projects/d28e4df6afda44cab21f12f0c2303b9a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:30.227343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf18c9b-8d91-43da-8772-798f9ac39d40 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.227622 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf18c9b-8d91-43da-8772-798f9ac39d40 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.227895 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf18c9b-8d91-43da-8772-798f9ac39d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.247095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bdf18c9b-8d91-43da-8772-798f9ac39d40 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:30.248342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bdf18c9b-8d91-43da-8772-798f9ac39d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:30.307493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1193/2407] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:30.315654 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d800-2351-4683-87b2-85b67d2cad69 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.315932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d800-2351-4683-87b2-85b67d2cad69 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.316209 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d800-2351-4683-87b2-85b67d2cad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.336648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e3f7d800-2351-4683-87b2-85b67d2cad69 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:30.337834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e3f7d800-2351-4683-87b2-85b67d2cad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:30.345784 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1215/2408] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:30.350079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf18c9b-8d91-43da-8772-798f9ac39d40 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.359914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf18c9b-8d91-43da-8772-798f9ac39d40 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.364910 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.365508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.365780 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.366047 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.366680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:30.366977 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.368606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.381174 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1194/2409] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:30.385943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d800-2351-4683-87b2-85b67d2cad69 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.395960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d800-2351-4683-87b2-85b67d2cad69 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.400639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.401236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.401519 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.401793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.402447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.404380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.425078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:30.437353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1216/2410] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:30 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:30.438438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.448155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413178-943a-4bb7-a960-b6a87ed453ac admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.511392 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.512036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.512240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.512521 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.513139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.515057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.538019 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1195/2411] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:30 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:30.542106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.552693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102fe8-1b03-4271-80b0-f3dd9aae4089 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.557416 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.558032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.558308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.558578 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.559234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.560938 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.570981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1217/2412] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:30.574818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.585626 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3778e-bc42-45ae-a973-00bb9cc9c981 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.590425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caf69a-3b83-42d7-93d1-7c98902c2748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.591015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caf69a-3b83-42d7-93d1-7c98902c2748 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.591324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caf69a-3b83-42d7-93d1-7c98902c2748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.591594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caf69a-3b83-42d7-93d1-7c98902c2748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.592217 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caf69a-3b83-42d7-93d1-7c98902c2748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.594412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caf69a-3b83-42d7-93d1-7c98902c2748 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.604811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1196/2413] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:30.608987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.618697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd47c0-f173-4141-8d63-5388847038e4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.623333 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.624002 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.624310 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.624581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] PATH_INFO: `/v3/projects/cdab0f4021294adca9c3bbd4ae377ae0/users/cf1c169ffa1a45f6b3e78592bbbc8a6a/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.640851 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdab0f4021294adca9c3bbd4ae377ae0, user_id=cf1c169ffa1a45f6b3e78592bbbc8a6a, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.643539 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.657432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1218/2414] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:30 2022] PUT /identity/v3/projects/cdab0f4021294adca9c3bbd4ae377ae0/users/cf1c169ffa1a45f6b3e78592bbbc8a6a/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:30.669283 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2aa20-2285-401d-a415-ed03a6e684d2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.669561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2aa20-2285-401d-a415-ed03a6e684d2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.669829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2aa20-2285-401d-a415-ed03a6e684d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.698331 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-daa2aa20-2285-401d-a415-ed03a6e684d2 None None] MFA Rules not processed for user `cf1c169ffa1a45f6b3e78592bbbc8a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:30.699619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-daa2aa20-2285-401d-a415-ed03a6e684d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:30.745795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:30.771761 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1197/2415] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:30.776861 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.853137 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab25d1-da0b-455e-9ef4-604842670913 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.858104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.858831 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.859174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.859411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.860032 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.861978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:30.891505 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1219/2416] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:30 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:30.895725 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2aa20-2285-401d-a415-ed03a6e684d2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:30.907033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2aa20-2285-401d-a415-ed03a6e684d2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:30.987043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:30.987687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:30.987993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:30.988253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:30.988886 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:30.991078 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:31.013526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1198/2417] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:30 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:31.018033 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:31.028421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77bc25-1276-47ec-9a0c-55d9dee0acd8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:31.033358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:31.034068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:31.034348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:31.034613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:31.035350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:31.036949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:31.047437 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1220/2418] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:31.052062 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:31.062675 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482e5b9-3622-4359-9afe-8f5ef0382473 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:31.067180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:31.067838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:31.068109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:31.068378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] PATH_INFO: `/v3/projects/f236f85536a94a0eb8ed1097d895330f/users/26d91e9edf754c8a8d8bfea176eac83e/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:31.083536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f236f85536a94a0eb8ed1097d895330f, user_id=26d91e9edf754c8a8d8bfea176eac83e, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:31.086171 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:31.099256 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1199/2419] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:31 2022] PUT /identity/v3/projects/f236f85536a94a0eb8ed1097d895330f/users/26d91e9edf754c8a8d8bfea176eac83e/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:31.103183 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:31.112885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763f3f9-2d3d-4782-bce4-ba557345ab7e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:31.167974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03318e1b-55bb-4e33-8fe2-23834ea37bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:31.168582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318e1b-55bb-4e33-8fe2-23834ea37bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:31.168857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318e1b-55bb-4e33-8fe2-23834ea37bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:31.169126 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318e1b-55bb-4e33-8fe2-23834ea37bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:31.169753 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03318e1b-55bb-4e33-8fe2-23834ea37bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:31.171363 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03318e1b-55bb-4e33-8fe2-23834ea37bd7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:31.181068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1221/2420] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:31.185175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:31.195867 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a7ecc-1aca-4e10-921d-403b8b64b90e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:31.200842 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:31.201473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:31.202342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:31.202595 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] PATH_INFO: `/v3/projects/f236f85536a94a0eb8ed1097d895330f/users/26d91e9edf754c8a8d8bfea176eac83e/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:31.204481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f236f85536a94a0eb8ed1097d895330f, user_id=26d91e9edf754c8a8d8bfea176eac83e, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:31.207096 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:31.216858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1200/2421] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:31 2022] PUT /identity/v3/projects/f236f85536a94a0eb8ed1097d895330f/users/26d91e9edf754c8a8d8bfea176eac83e/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:31.226776 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe276fd-7bf4-4a9b-9177-d0489bc0856f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:31.227055 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe276fd-7bf4-4a9b-9177-d0489bc0856f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:31.227359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe276fd-7bf4-4a9b-9177-d0489bc0856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:31.255685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dbe276fd-7bf4-4a9b-9177-d0489bc0856f None None] MFA Rules not processed for user `26d91e9edf754c8a8d8bfea176eac83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:31.256863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dbe276fd-7bf4-4a9b-9177-d0489bc0856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:31.276036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:31.324410 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1222/2422] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:31.337533 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:31.339264 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:31.396391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ad281-361c-4414-aeae-950a3f57707c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:31.400243 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:31.400841 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:31.401139 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:31.401403 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:31.410394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:31.411868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:31.421709 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1201/2423] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:31.493998 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe276fd-7bf4-4a9b-9177-d0489bc0856f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:31.504317 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe276fd-7bf4-4a9b-9177-d0489bc0856f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:31.507911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc440a0-05b3-4a17-8484-3b68291bf11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:31.508527 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc440a0-05b3-4a17-8484-3b68291bf11f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:31.508798 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc440a0-05b3-4a17-8484-3b68291bf11f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:31.509065 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc440a0-05b3-4a17-8484-3b68291bf11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:31.565038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc440a0-05b3-4a17-8484-3b68291bf11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:31.566488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc440a0-05b3-4a17-8484-3b68291bf11f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:31.575987 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1223/2424] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:34.361234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:34.406155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:34.432150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f46f-bf62-4c86-bdfd-03d6b63cfc36 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:34.438102 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:34.438813 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:34.439120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:34.439399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] PATH_INFO: `/v3/users/cf1c169ffa1a45f6b3e78592bbbc8a6a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:34.440551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1c169ffa1a45f6b3e78592bbbc8a6a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:34.442547 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:34.509732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1c169ffa1a45f6b3e78592bbbc8a6a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.510911 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1c169ffa1a45f6b3e78592bbbc8a6a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.520388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1c169ffa1a45f6b3e78592bbbc8a6a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.529942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1c169ffa1a45f6b3e78592bbbc8a6a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.530712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1c169ffa1a45f6b3e78592bbbc8a6a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.558499 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1202/2425] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:34 2022] DELETE /identity/v3/users/cf1c169ffa1a45f6b3e78592bbbc8a6a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:34.560190 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:34.560685 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:34.560830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:34.582177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:34.583431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:34.677080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1224/2426] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:34.687286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:34.688892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:34.747382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09365e-629a-4e9f-9e58-c19bdfa831bc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:34.751106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:34.751741 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:34.752024 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:34.752299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:34.762918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:34.764643 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:34.774388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1203/2427] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:34.841688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:34.843764 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:34.856421 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19f269-6056-4e5b-af82-051d7a73cc73 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:34.861773 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:34.862394 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:34.862666 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:34.863020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] PATH_INFO: `/v3/users/e6a48b2767514860bc5d7481f7e3ab87` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:34.864234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a48b2767514860bc5d7481f7e3ab87)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:34.866457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:34.915718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a48b2767514860bc5d7481f7e3ab87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.916488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a48b2767514860bc5d7481f7e3ab87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.922697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:34.926951 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a48b2767514860bc5d7481f7e3ab87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.928000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a48b2767514860bc5d7481f7e3ab87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.942251 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a48b2767514860bc5d7481f7e3ab87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:34.960690 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1225/2428] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:34 2022] DELETE /identity/v3/users/e6a48b2767514860bc5d7481f7e3ab87 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:35.076142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:35.078237 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:35.097947 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aec33e-2125-4874-8049-18f5db51582d service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:35.166770 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:35.168587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:35.183079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:35.183963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:35.184242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:35.184527 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] PATH_INFO: `/v3/projects/cdab0f4021294adca9c3bbd4ae377ae0` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:35.185659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdab0f4021294adca9c3bbd4ae377ae0)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:35.187916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:35.189508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:35.239575 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b01d8d-908d-49da-bf31-633f16e5f901 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:35.240457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdab0f4021294adca9c3bbd4ae377ae0', 'request_id': 'req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.246076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdab0f4021294adca9c3bbd4ae377ae0', 'request_id': 'req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.254161 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdab0f4021294adca9c3bbd4ae377ae0', 'request_id': 'req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.266244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] The token cache is being invalidate because project cdab0f4021294adca9c3bbd4ae377ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:35.266951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.267605 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.269257 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1204/2429] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:35 2022] DELETE /identity/v3/projects/cdab0f4021294adca9c3bbd4ae377ae0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:35.273201 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:35.274394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:35.298019 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:35.298648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:35.298927 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:35.299199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] PATH_INFO: `/v3/projects/c5fc51cd319448d5a83b6038c4610d61` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:35.300353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5fc51cd319448d5a83b6038c4610d61)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:35.302271 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:35.338956 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146214eb-3fc0-47c5-ab05-2ed39e8a8c6b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:35.348503 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5fc51cd319448d5a83b6038c4610d61', 'request_id': 'req-eb2f1eab-38ed-4027-8a59-2560574a161b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.354157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5fc51cd319448d5a83b6038c4610d61', 'request_id': 'req-eb2f1eab-38ed-4027-8a59-2560574a161b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.361295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5fc51cd319448d5a83b6038c4610d61', 'request_id': 'req-eb2f1eab-38ed-4027-8a59-2560574a161b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.374887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] The token cache is being invalidate because project c5fc51cd319448d5a83b6038c4610d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:35.375171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.375817 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.377306 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1226/2430] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:35 2022] DELETE /identity/v3/projects/c5fc51cd319448d5a83b6038c4610d61 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:35.381050 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:35.382353 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:35.391084 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:35.398699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:35.399362 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:35.399637 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:35.399906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] PATH_INFO: `/v3/users/26d91e9edf754c8a8d8bfea176eac83e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:35.401044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26d91e9edf754c8a8d8bfea176eac83e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:35.403369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:35.445092 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:35.451404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f1eab-38ed-4027-8a59-2560574a161b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:35.453359 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26d91e9edf754c8a8d8bfea176eac83e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.454066 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26d91e9edf754c8a8d8bfea176eac83e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.456682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:35.457334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:35.457637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:35.457911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] PATH_INFO: `/v3/users/714daf22c961443391cb43bedd66b3d2` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:35.459026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714daf22c961443391cb43bedd66b3d2)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:35.460972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:35.461823 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26d91e9edf754c8a8d8bfea176eac83e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.469336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26d91e9edf754c8a8d8bfea176eac83e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.470159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26d91e9edf754c8a8d8bfea176eac83e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.488551 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1205/2431] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:35 2022] DELETE /identity/v3/users/26d91e9edf754c8a8d8bfea176eac83e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:35.510863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714daf22c961443391cb43bedd66b3d2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.511537 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714daf22c961443391cb43bedd66b3d2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.519923 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714daf22c961443391cb43bedd66b3d2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.520830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714daf22c961443391cb43bedd66b3d2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.534446 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714daf22c961443391cb43bedd66b3d2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.547721 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1227/2432] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:35 2022] DELETE /identity/v3/users/714daf22c961443391cb43bedd66b3d2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:35.729266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:35.734501 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:35.779187 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:35.780605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:35.840753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64589e-434b-48d7-a9d0-a964b946661c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:35.872769 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ec5-aa23-4a7f-80d6-56b7ae6a7ee6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:35.878801 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:35.879461 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:35.879745 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:35.880020 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] PATH_INFO: `/v3/projects/f236f85536a94a0eb8ed1097d895330f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:35.881272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f236f85536a94a0eb8ed1097d895330f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:35.883181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:35.914294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:35.915121 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:35.915423 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:35.915763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] PATH_INFO: `/v3/projects/5096d5eb5cc341ccb47850ae73ef8e6e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:35.917088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5096d5eb5cc341ccb47850ae73ef8e6e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:35.919149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:35.925869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:35.953660 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f236f85536a94a0eb8ed1097d895330f', 'request_id': 'req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.960220 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f236f85536a94a0eb8ed1097d895330f', 'request_id': 'req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.967972 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f236f85536a94a0eb8ed1097d895330f', 'request_id': 'req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.981234 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5096d5eb5cc341ccb47850ae73ef8e6e', 'request_id': 'req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.981909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] The token cache is being invalidate because project f236f85536a94a0eb8ed1097d895330f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:35.982300 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.983487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be0e2d45-bbd4-4a8c-aaa4-a7f454927aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.985671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1228/2433] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:35 2022] DELETE /identity/v3/projects/f236f85536a94a0eb8ed1097d895330f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:35.987948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5096d5eb5cc341ccb47850ae73ef8e6e', 'request_id': 'req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.990181 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220a75c-b2e8-4a63-8845-eb90e080719e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:35.990529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220a75c-b2e8-4a63-8845-eb90e080719e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:35.990858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220a75c-b2e8-4a63-8845-eb90e080719e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:35.996620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5096d5eb5cc341ccb47850ae73ef8e6e', 'request_id': 'req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:35.997831 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:36.012399 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f220a75c-b2e8-4a63-8845-eb90e080719e None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:36.013665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f220a75c-b2e8-4a63-8845-eb90e080719e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.016624 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] The token cache is being invalidate because project 5096d5eb5cc341ccb47850ae73ef8e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:36.017487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:36.018266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a6a0709-fc2b-4cc9-b7f2-c28bc24129c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:36.020304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1206/2434] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:35 2022] DELETE /identity/v3/projects/5096d5eb5cc341ccb47850ae73ef8e6e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:36.024587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f430785-3468-44ab-a879-c22655a4ed65 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.025093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f430785-3468-44ab-a879-c22655a4ed65 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.025396 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f430785-3468-44ab-a879-c22655a4ed65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.048084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3f430785-3468-44ab-a879-c22655a4ed65 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:36.049315 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3f430785-3468-44ab-a879-c22655a4ed65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.089145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1229/2435] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:36.098679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1619fbe-83b0-446b-a00c-01414a9eaff4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.098679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1619fbe-83b0-446b-a00c-01414a9eaff4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.098679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1619fbe-83b0-446b-a00c-01414a9eaff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.129177 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b1619fbe-83b0-446b-a00c-01414a9eaff4 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:36.130580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b1619fbe-83b0-446b-a00c-01414a9eaff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.139995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1207/2436] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:36.148413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc546c8-a4eb-4e85-94ee-bfc563e5ef31 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.148753 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc546c8-a4eb-4e85-94ee-bfc563e5ef31 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.149027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc546c8-a4eb-4e85-94ee-bfc563e5ef31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.155659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1230/2437] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:36.157146 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1619fbe-83b0-446b-a00c-01414a9eaff4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.167517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1619fbe-83b0-446b-a00c-01414a9eaff4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.174935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.175580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.175858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.179250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.179250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:36.179250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.179250 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.187782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8dc546c8-a4eb-4e85-94ee-bfc563e5ef31 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:36.189748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8dc546c8-a4eb-4e85-94ee-bfc563e5ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.193056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1231/2438] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:36.197442 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.199173 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1208/2439] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:36.203486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc546c8-a4eb-4e85-94ee-bfc563e5ef31 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.213127 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b83d6c-ef65-413e-acda-0675c0fed710 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.213303 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc546c8-a4eb-4e85-94ee-bfc563e5ef31 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.218900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.219048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.219506 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.219689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.219796 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.220053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.220165 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.220418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.220816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.221397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:36.221856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.222763 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.223905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.237987 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1209/2440] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:36.243011 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.253562 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1232/2441] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:36.256941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e36bd-8d81-40c0-85ca-163c25f32b17 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.257708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.310487 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.311115 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.311492 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.311719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.312413 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.314297 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.319029 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92321283-db24-45af-bce3-e72dbcd5bb52 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.324022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.324591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.324870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.325393 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.326010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.327958 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.342661 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1210/2442] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:36.347095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.353486 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1233/2443] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:36.354674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.361212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:36.366713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d379-a957-4183-b860-263981aa6507 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.409666 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3caf7-605c-4544-bb22-4dd0e1ea0f4b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.414300 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.414889 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.415164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.415471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.416104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.418382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.427122 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:36.429862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.430454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.430729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.431000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.431815 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.433662 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.443678 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1211/2444] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:36.447579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1234/2445] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:36.447934 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.451834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.459477 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03961b8-41b6-4a12-9ca2-91e5217bd5e8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.461912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e34fc3-b660-4618-8ec6-12767c3358d4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.464438 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.465028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.465336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.465608 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.466233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.466543 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.467679 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.467950 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.468222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.468384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.468872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.470477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.478764 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1212/2446] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:36.484111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1235/2447] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:36.485221 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.485351 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.495390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c040c9a-3e59-4186-a020-b313dd9bcbed admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.496107 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be251ab-49c7-4ec2-b17f-69cf34708425 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.500109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.500872 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.501004 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244149-b2b5-4917-be21-d6c2d79b7bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.501155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.501431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.501631 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244149-b2b5-4917-be21-d6c2d79b7bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.501812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244149-b2b5-4917-be21-d6c2d79b7bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.502080 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244149-b2b5-4917-be21-d6c2d79b7bd8 admin admin] PATH_INFO: `/v3/projects/6034fe4211af489e8ed213dd2c788d88/users/d137c46e0e8046e7beddcf3845e71541/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.502258 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.503990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.515811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1213/2448] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:36.518392 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244149-b2b5-4917-be21-d6c2d79b7bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6034fe4211af489e8ed213dd2c788d88, user_id=d137c46e0e8046e7beddcf3845e71541, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.519850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.521352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244149-b2b5-4917-be21-d6c2d79b7bd8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.530694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d32c9-c28e-48a0-ab64-ad561c84c32f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.534674 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1236/2449] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:36 2022] PUT /identity/v3/projects/6034fe4211af489e8ed213dd2c788d88/users/d137c46e0e8046e7beddcf3845e71541/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:36.545990 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.546272 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.546544 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.574179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] MFA Rules not processed for user `d137c46e0e8046e7beddcf3845e71541`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:36.575687 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.588305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e39c-122f-45e4-9758-21900e47c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.588955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e39c-122f-45e4-9758-21900e47c228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.589227 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e39c-122f-45e4-9758-21900e47c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.589496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e39c-122f-45e4-9758-21900e47c228 admin admin] PATH_INFO: `/v3/projects/25a832dffd5446ecac869867d94186ac/users/4f7007a6d0b24d3e8a6148a64a05dbe9/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.604132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e39c-122f-45e4-9758-21900e47c228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a832dffd5446ecac869867d94186ac, user_id=4f7007a6d0b24d3e8a6148a64a05dbe9, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.606757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e39c-122f-45e4-9758-21900e47c228 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.621405 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1214/2450] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:36 2022] PUT /identity/v3/projects/25a832dffd5446ecac869867d94186ac/users/4f7007a6d0b24d3e8a6148a64a05dbe9/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:36.633079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.633358 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.633631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.642012 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1237/2451] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:36.652860 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.654215 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.662095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] MFA Rules not processed for user `4f7007a6d0b24d3e8a6148a64a05dbe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:36.663316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.711103 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003294e-173f-4cf0-98bb-7f08e102efea None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.719244 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:36.729494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1215/2452] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:36.741760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:36.743126 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:36.760298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3db220-3faa-4bab-aff7-90d8551d1b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.761282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db220-3faa-4bab-aff7-90d8551d1b80 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.761282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db220-3faa-4bab-aff7-90d8551d1b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.761533 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db220-3faa-4bab-aff7-90d8551d1b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.786649 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:36.802354 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4078c-c580-4b92-8f4c-718396c75814 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:36.806198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:36.806794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:36.807069 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:36.807376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:36.816089 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.817560 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.819748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3db220-3faa-4bab-aff7-90d8551d1b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:36.821212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3db220-3faa-4bab-aff7-90d8551d1b80 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:36.827025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1216/2453] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:36.832868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1238/2454] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:45.678366 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:45.679903 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:45.746520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c74ac-ccae-4a06-ad54-23be6923b3ad service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:45.751623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:45.752246 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:45.752523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:45.752795 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] PATH_INFO: `/v3/users/e267c52c0e5e4c09b92db4a85147033d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:45.753949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e267c52c0e5e4c09b92db4a85147033d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:45.755906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:45.802924 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e267c52c0e5e4c09b92db4a85147033d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:45.803573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e267c52c0e5e4c09b92db4a85147033d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:45.811007 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e267c52c0e5e4c09b92db4a85147033d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:45.819180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e267c52c0e5e4c09b92db4a85147033d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:45.819799 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e267c52c0e5e4c09b92db4a85147033d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:45.837826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1217/2455] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:45 2022] DELETE /identity/v3/users/e267c52c0e5e4c09b92db4a85147033d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:45.842329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:45.842809 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:45.842911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:45.862268 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:45.863478 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:45.920833 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1239/2456] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:45.931259 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:45.932674 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:45.989437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feff8d8-e0a2-425b-9217-07c54136e6c5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:45.992995 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:45.993580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:45.993878 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:45.994145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:46.002677 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:46.004127 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:46.013242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1218/2457] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:46.269529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:46.271153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.284090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e5f0e-be84-4820-b4c3-b15beb3b1f96 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:46.288971 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:46.289457 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:46.289735 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:46.290015 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] PATH_INFO: `/v3/users/d137c46e0e8046e7beddcf3845e71541` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:46.291383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d137c46e0e8046e7beddcf3845e71541)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:46.292393 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:46.293246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:46.294055 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.306897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:46.311029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf44897-c109-45dc-ad8c-8910f64c76cd service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:46.319379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:46.320693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:46.321238 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:46.322094 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] PATH_INFO: `/v3/projects/03ce305ca2f54883947c3ec2c8c4d498` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:46.323838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ce305ca2f54883947c3ec2c8c4d498)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:46.326543 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:46.352122 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd137c46e0e8046e7beddcf3845e71541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.352988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd137c46e0e8046e7beddcf3845e71541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.360498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd137c46e0e8046e7beddcf3845e71541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.361299 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd137c46e0e8046e7beddcf3845e71541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.367303 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:46.376984 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c27baadf-ad59-4c36-9a2f-b64b9396599a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd137c46e0e8046e7beddcf3845e71541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.380604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ce305ca2f54883947c3ec2c8c4d498', 'request_id': 'req-e4278b27-c3bd-4657-87b8-1dbdf9908f55'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.387117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ce305ca2f54883947c3ec2c8c4d498', 'request_id': 'req-e4278b27-c3bd-4657-87b8-1dbdf9908f55'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.389841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1240/2458] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:46 2022] DELETE /identity/v3/users/d137c46e0e8046e7beddcf3845e71541 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:46.394821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:46.395174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:46.395479 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:46.396060 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ce305ca2f54883947c3ec2c8c4d498', 'request_id': 'req-e4278b27-c3bd-4657-87b8-1dbdf9908f55'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.409289 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] The token cache is being invalidate because project 03ce305ca2f54883947c3ec2c8c4d498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:46.409575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.410323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.412394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1219/2459] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:46 2022] DELETE /identity/v3/projects/03ce305ca2f54883947c3ec2c8c4d498 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:46.416206 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:46.417136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:46.417727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.418390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.483710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1241/2460] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:46.487918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4278b27-c3bd-4657-87b8-1dbdf9908f55 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:46.492918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:46.493231 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:46.493545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:46.493828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:46.494100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] PATH_INFO: `/v3/users/3eb11773c4d84cc5a338c138995cea5f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:46.494748 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.495233 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb11773c4d84cc5a338c138995cea5f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:46.497154 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:46.543236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb11773c4d84cc5a338c138995cea5f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.544175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb11773c4d84cc5a338c138995cea5f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.551846 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb11773c4d84cc5a338c138995cea5f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.554426 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107c54d-b8b0-47cc-8332-5ad815c83045 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:46.558886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:46.559579 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:46.559858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:46.559982 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb11773c4d84cc5a338c138995cea5f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.560185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:46.560603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb11773c4d84cc5a338c138995cea5f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:46.562066 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.574349 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:46.576045 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:46.578726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1220/2461] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:46 2022] DELETE /identity/v3/users/3eb11773c4d84cc5a338c138995cea5f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:46.588659 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1242/2462] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:46.943645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:46.945602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.969920 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:46.971420 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:46.972722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:46.990197 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4ccb4-b05f-4d6e-8f78-8c8bc8ac3fc9 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.047316 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d292953-f9bc-4fbc-a988-b0c011a269d9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.050982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.051705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.051844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.052069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.052346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] PATH_INFO: `/v3/projects/6034fe4211af489e8ed213dd2c788d88` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.052454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.052717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.052984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] PATH_INFO: `/v3/projects/9190b3ab69d04ff88a902d3cba24d1a2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.053464 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6034fe4211af489e8ed213dd2c788d88)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.054093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9190b3ab69d04ff88a902d3cba24d1a2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.055397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.056002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.082961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:47.103919 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9190b3ab69d04ff88a902d3cba24d1a2', 'request_id': 'req-0742f173-fb07-441d-8799-2407f61317a8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.106605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6034fe4211af489e8ed213dd2c788d88', 'request_id': 'req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.110048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9190b3ab69d04ff88a902d3cba24d1a2', 'request_id': 'req-0742f173-fb07-441d-8799-2407f61317a8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.113235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6034fe4211af489e8ed213dd2c788d88', 'request_id': 'req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.117620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9190b3ab69d04ff88a902d3cba24d1a2', 'request_id': 'req-0742f173-fb07-441d-8799-2407f61317a8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.119941 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6034fe4211af489e8ed213dd2c788d88', 'request_id': 'req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.130133 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] The token cache is being invalidate because project 9190b3ab69d04ff88a902d3cba24d1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:47.130428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.131083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0742f173-fb07-441d-8799-2407f61317a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.132669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1221/2463] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:46 2022] DELETE /identity/v3/projects/9190b3ab69d04ff88a902d3cba24d1a2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:47.133254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] The token cache is being invalidate because project 6034fe4211af489e8ed213dd2c788d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:47.133546 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.134174 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.147236 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1243/2464] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:46 2022] DELETE /identity/v3/projects/6034fe4211af489e8ed213dd2c788d88 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:47.147502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691a9c3-19ae-40b8-8224-998bbe060087 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.147502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691a9c3-19ae-40b8-8224-998bbe060087 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.147502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691a9c3-19ae-40b8-8224-998bbe060087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.160388 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5691a9c3-19ae-40b8-8224-998bbe060087 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:47.161630 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5691a9c3-19ae-40b8-8224-998bbe060087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.192684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.193760 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.231757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1222/2465] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:47.240436 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f2792-668f-4f10-b01a-ae394b84652d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.240735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f2792-668f-4f10-b01a-ae394b84652d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.241407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f2792-668f-4f10-b01a-ae394b84652d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.262404 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-381f2792-668f-4f10-b01a-ae394b84652d None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:47.263665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-381f2792-668f-4f10-b01a-ae394b84652d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.272052 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c23bd-c0a1-4d1f-9e81-377314b89ee3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.272323 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1223/2466] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:47.276752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f2792-668f-4f10-b01a-ae394b84652d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.277176 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.277828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.278105 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.278380 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] PATH_INFO: `/v3/users/4f7007a6d0b24d3e8a6148a64a05dbe9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.279585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7007a6d0b24d3e8a6148a64a05dbe9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.281608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.287459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f2792-668f-4f10-b01a-ae394b84652d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.292439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.293268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.294056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.294056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.294732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:47.295130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.297692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.315545 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1224/2467] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 07:11:47.319690 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.331803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7372d-865e-4d2e-a911-853883ba4730 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.336860 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.337473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.337744 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.338014 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.338640 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.340677 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.341581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7007a6d0b24d3e8a6148a64a05dbe9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.342398 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7007a6d0b24d3e8a6148a64a05dbe9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.351371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7007a6d0b24d3e8a6148a64a05dbe9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.352725 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7007a6d0b24d3e8a6148a64a05dbe9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.360887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:47.370511 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bcdd1db7-c3b2-48b7-9fcb-6483447a9e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7007a6d0b24d3e8a6148a64a05dbe9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.377853 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1225/2468] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:47.381991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.383802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.384693 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1244/2469] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:47 2022] DELETE /identity/v3/users/4f7007a6d0b24d3e8a6148a64a05dbe9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:47.388965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.389254 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.389529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.408880 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:47.410075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.436480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:47.452286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb4c7-04ef-4702-9ced-b26ee2e179b2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.472339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1245/2470] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:47.482413 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.483908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.508535 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.509145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.509424 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.509696 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.510635 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.512284 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.535301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1226/2471] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:47.539103 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.542703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052358-4aef-4229-9e46-0acb37725a45 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.546290 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.546909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.547182 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.547476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.548730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c957c-15d3-498e-a93e-37edc075564d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.553478 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.554567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.554873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.555146 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.555859 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.556477 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.557454 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.558171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.570713 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1227/2472] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:47.570912 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1246/2473] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:47.572100 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.582510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588715a2-b44d-4f88-ab0d-502b5246d2a6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.587368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3fe65-0920-4ef2-bf2c-9204057c9544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.587950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3fe65-0920-4ef2-bf2c-9204057c9544 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.588286 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3fe65-0920-4ef2-bf2c-9204057c9544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.588565 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3fe65-0920-4ef2-bf2c-9204057c9544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.589236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3fe65-0920-4ef2-bf2c-9204057c9544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.590916 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3fe65-0920-4ef2-bf2c-9204057c9544 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.604386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1228/2474] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:47.605538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.615384 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae8ecb-da57-4471-9648-b04eb465bb0f service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.620883 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.621600 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.621842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.622113 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] PATH_INFO: `/v3/projects/ee98dea90e1a48d8b1f72bdfbc341b88/users/23d20107cee445b09a56443cbab4c903/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.639148 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee98dea90e1a48d8b1f72bdfbc341b88, user_id=23d20107cee445b09a56443cbab4c903, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.642701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.657788 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1247/2475] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:47 2022] PUT /identity/v3/projects/ee98dea90e1a48d8b1f72bdfbc341b88/users/23d20107cee445b09a56443cbab4c903/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:47.668379 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.668688 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.668958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.697530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] MFA Rules not processed for user `23d20107cee445b09a56443cbab4c903`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:47.698732 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.778279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:47.782788 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1229/2476] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:47.787160 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.850962 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcccc2ba-da78-49bb-b2a4-427cf7b55481 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.855743 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.856337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.856617 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.856886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.857606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.859580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.860067 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.861168 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:47.878474 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cefddf-08eb-4265-9c30-b76d531abc62 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.889327 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1248/2477] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:47.890511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:47.891153 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.891465 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.891739 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] PATH_INFO: `/v3/projects/25a832dffd5446ecac869867d94186ac` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.892967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a832dffd5446ecac869867d94186ac)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:47.893273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:47.895408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:47.932217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:47.946835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a832dffd5446ecac869867d94186ac', 'request_id': 'req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.953794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a832dffd5446ecac869867d94186ac', 'request_id': 'req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.959049 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d73cf-fb7c-464d-bfd2-a715430fe862 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:47.961699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a832dffd5446ecac869867d94186ac', 'request_id': 'req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.972950 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] The token cache is being invalidate because project 25a832dffd5446ecac869867d94186ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:47.973232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.973875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cacc0d0-dc34-4d1f-953a-d4e45dd8a0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:47.975647 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1230/2478] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:47 2022] DELETE /identity/v3/projects/25a832dffd5446ecac869867d94186ac => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:47.979999 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fc94e-6cca-4f75-906a-6cdcec033f09 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:47.980275 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fc94e-6cca-4f75-906a-6cdcec033f09 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:47.980548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fc94e-6cca-4f75-906a-6cdcec033f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:47.999887 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6a1fc94e-6cca-4f75-906a-6cdcec033f09 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:48.001086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a1fc94e-6cca-4f75-906a-6cdcec033f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:48.016106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.016713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.016989 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.017260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.017881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.019820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.043316 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1249/2479] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:48.045550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.047389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:48.058667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06d46f-3935-4217-8abd-eab7dd1926d4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.063794 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.064376 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.065203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.065203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.065536 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.067111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.067897 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1231/2480] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:48.076082 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091140d6-e75e-4cb7-8a16-938eb9cca4f1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.076368 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091140d6-e75e-4cb7-8a16-938eb9cca4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.076644 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091140d6-e75e-4cb7-8a16-938eb9cca4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.080793 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1250/2481] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:48.081697 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.091125 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae3d01-b986-41b1-9080-0299779cc781 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.095717 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.096383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.096664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.096935 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] PATH_INFO: `/v3/projects/1f4ec586270b47fc92e785f31c2ed702/users/ecaedf374ec94c36b6ba25121b81d357/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.097148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-091140d6-e75e-4cb7-8a16-938eb9cca4f1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:48.098343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-091140d6-e75e-4cb7-8a16-938eb9cca4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:48.106216 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1232/2482] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:48.110946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091140d6-e75e-4cb7-8a16-938eb9cca4f1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.114285 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4ec586270b47fc92e785f31c2ed702, user_id=ecaedf374ec94c36b6ba25121b81d357, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.116981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.120299 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091140d6-e75e-4cb7-8a16-938eb9cca4f1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.126029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.126639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.126923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.127266 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.128108 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:48.128336 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.129951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.132816 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1251/2483] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:48 2022] PUT /identity/v3/projects/1f4ec586270b47fc92e785f31c2ed702/users/ecaedf374ec94c36b6ba25121b81d357/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:48.136762 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.143352 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1233/2484] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:48.151364 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.161592 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecfe5c-ce1e-4208-a93e-364749e82459 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.199403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ddfc-d107-4828-a367-f5f741dbe744 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.205325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.205920 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.206196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.206470 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.207099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.208711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.218681 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1252/2485] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:48.219824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.220425 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.220699 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.220969 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.221742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.222488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.223662 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.231821 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124099ad-2c6e-4b22-a669-f7bc4bed2bd5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.233067 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:48.236701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c2b1-bf9c-471b-8b37-1b0bcf5be312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.237368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c2b1-bf9c-471b-8b37-1b0bcf5be312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.237667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c2b1-bf9c-471b-8b37-1b0bcf5be312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.237946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c2b1-bf9c-471b-8b37-1b0bcf5be312 admin admin] PATH_INFO: `/v3/projects/1f4ec586270b47fc92e785f31c2ed702/users/ecaedf374ec94c36b6ba25121b81d357/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.239884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4c2b1-bf9c-471b-8b37-1b0bcf5be312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4ec586270b47fc92e785f31c2ed702, user_id=ecaedf374ec94c36b6ba25121b81d357, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.242811 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4c2b1-bf9c-471b-8b37-1b0bcf5be312 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.256596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1234/2486] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:48 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:48.256779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1253/2487] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:48 2022] PUT /identity/v3/projects/1f4ec586270b47fc92e785f31c2ed702/users/ecaedf374ec94c36b6ba25121b81d357/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:48.258048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.265090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.266069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.266344 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.295893 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] MFA Rules not processed for user `ecaedf374ec94c36b6ba25121b81d357`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:48.297041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:48.305460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:48.323415 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72d4a4-efde-47a2-98dd-75f512d354bb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.328803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.329394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.329667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.329935 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.330544 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.332502 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.353518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1235/2488] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:48 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:48.357792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.367104 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108382-9915-48e3-82b4-93d41e13ab90 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.371555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1254/2489] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:48.371826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.372471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.372747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.373018 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.373736 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.376046 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.382933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.384463 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:48.389058 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1236/2490] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:48.390015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.401131 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe2861-0f86-47d3-8091-f91b4c3200de admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.405789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.406376 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.406652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.406923 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.407575 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.409189 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.421994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1237/2491] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:48.422944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.432496 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1d9f-8ea3-4280-8a27-937e05df6a5f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.437902 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fbe5e-7cbf-45fd-bcbc-3da9526f29bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.438529 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fbe5e-7cbf-45fd-bcbc-3da9526f29bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.438803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fbe5e-7cbf-45fd-bcbc-3da9526f29bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.439072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fbe5e-7cbf-45fd-bcbc-3da9526f29bf admin admin] PATH_INFO: `/v3/projects/27c9c9c8efd64131b5944c5cf47553b3/users/2dda9b6c5edd4dec87e4d23ecfcea5e9/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.444668 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d00f8-62ab-4a25-b123-13cc93a04c1c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.448296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.448887 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.449159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.449483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.454464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fbe5e-7cbf-45fd-bcbc-3da9526f29bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c9c9c8efd64131b5944c5cf47553b3, user_id=2dda9b6c5edd4dec87e4d23ecfcea5e9, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.457155 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fbe5e-7cbf-45fd-bcbc-3da9526f29bf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.460008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.465222 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.474461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1238/2492] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:48 2022] PUT /identity/v3/projects/27c9c9c8efd64131b5944c5cf47553b3/users/2dda9b6c5edd4dec87e4d23ecfcea5e9/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:48.475664 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1255/2493] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:48.480567 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481601f-4c75-4856-bfd8-201ce4167db1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.480849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481601f-4c75-4856-bfd8-201ce4167db1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.481159 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481601f-4c75-4856-bfd8-201ce4167db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.501093 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1481601f-4c75-4856-bfd8-201ce4167db1 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:48.502536 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1481601f-4c75-4856-bfd8-201ce4167db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:48.572562 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1239/2494] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:48.583959 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:48.588675 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:48.652868 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f71be-2d05-4f64-af09-56c000aa466f service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:48.702143 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a1a39-c5ff-4dc6-a671-31488829913b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:48.703293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a1a39-c5ff-4dc6-a671-31488829913b service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:48.703568 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a1a39-c5ff-4dc6-a671-31488829913b service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:48.703834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a1a39-c5ff-4dc6-a671-31488829913b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:48.714188 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a1a39-c5ff-4dc6-a671-31488829913b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:48.716273 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a1a39-c5ff-4dc6-a671-31488829913b service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:48.726062 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1256/2495] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:49.936786 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481601f-4c75-4856-bfd8-201ce4167db1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:49.948106 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481601f-4c75-4856-bfd8-201ce4167db1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:49.955468 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:49.956209 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:49.956683 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:49.957175 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] PATH_INFO: `/v3/users/23d20107cee445b09a56443cbab4c903` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:49.958574 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d20107cee445b09a56443cbab4c903)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:49.960931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:49.999564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:50.022806 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d20107cee445b09a56443cbab4c903'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.023505 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d20107cee445b09a56443cbab4c903'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.032986 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d20107cee445b09a56443cbab4c903'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.042172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d20107cee445b09a56443cbab4c903'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.042951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d20107cee445b09a56443cbab4c903'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.064648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1240/2496] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:49 2022] DELETE /identity/v3/users/23d20107cee445b09a56443cbab4c903 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:50.071051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:50.072198 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:50.072514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:50.091000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:50.100964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:50.102256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:50.200621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1257/2497] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:50.211790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:50.213390 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:50.290342 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e2a60-3467-4a72-a82f-b0b7f57d6cef admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:50.309727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:50.310432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:50.310623 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:50.310901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:50.321361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:50.322730 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:50.332927 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1241/2498] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:50.651159 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:50.653028 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:50.665248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e413ff-58ca-42de-8102-eb5c3c1e9af6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:50.670718 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:50.672051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:50.672400 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:50.672744 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] PATH_INFO: `/v3/projects/ee98dea90e1a48d8b1f72bdfbc341b88` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:50.673943 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee98dea90e1a48d8b1f72bdfbc341b88)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:50.676035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:50.724021 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee98dea90e1a48d8b1f72bdfbc341b88', 'request_id': 'req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.729635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee98dea90e1a48d8b1f72bdfbc341b88', 'request_id': 'req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.736447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee98dea90e1a48d8b1f72bdfbc341b88', 'request_id': 'req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.749180 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] The token cache is being invalidate because project ee98dea90e1a48d8b1f72bdfbc341b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:50.749525 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.750652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.752385 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1258/2499] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:50 2022] DELETE /identity/v3/projects/ee98dea90e1a48d8b1f72bdfbc341b88 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:50.756185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:50.768279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0df241-c2f6-4295-8c4c-2d1d5c5d6f6c service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:50.840790 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:50.841479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:50.841709 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:50.841966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] PATH_INFO: `/v3/users/ecaedf374ec94c36b6ba25121b81d357` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:50.843076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecaedf374ec94c36b6ba25121b81d357)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:50.845572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:50.906561 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecaedf374ec94c36b6ba25121b81d357'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.907601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecaedf374ec94c36b6ba25121b81d357'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.912135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:50.918378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecaedf374ec94c36b6ba25121b81d357'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.928134 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecaedf374ec94c36b6ba25121b81d357'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.928829 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f5709428-8f55-450d-a4bb-a2eaf0cd4803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecaedf374ec94c36b6ba25121b81d357'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:50.953873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1242/2500] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:50 2022] DELETE /identity/v3/users/ecaedf374ec94c36b6ba25121b81d357 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:51.324185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:51.326626 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:51.353870 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fc6af-8a3b-452e-b9b2-56f9af3ed1a0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:51.447193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:51.448350 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:51.448933 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:51.449352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] PATH_INFO: `/v3/projects/1f4ec586270b47fc92e785f31c2ed702` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:51.451569 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4ec586270b47fc92e785f31c2ed702)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:51.455358 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:51.522401 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:51.544247 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4ec586270b47fc92e785f31c2ed702', 'request_id': 'req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:51.557240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4ec586270b47fc92e785f31c2ed702', 'request_id': 'req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:51.567596 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4ec586270b47fc92e785f31c2ed702', 'request_id': 'req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:51.585605 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] The token cache is being invalidate because project 1f4ec586270b47fc92e785f31c2ed702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:51.585944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:51.586665 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ca9cb96a-aa1a-40ec-baf6-eb0517c78cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:51.588835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1259/2501] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:51 2022] DELETE /identity/v3/projects/1f4ec586270b47fc92e785f31c2ed702 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:51.594193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c32e-cd45-4ff3-b786-4edad9b500b9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:51.594472 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c32e-cd45-4ff3-b786-4edad9b500b9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:51.594743 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c32e-cd45-4ff3-b786-4edad9b500b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:51.632112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7ad7c32e-cd45-4ff3-b786-4edad9b500b9 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:51.633001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ad7c32e-cd45-4ff3-b786-4edad9b500b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:51.742450 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1243/2502] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:51.744889 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ed3c4-0ee2-492c-80bc-85c17ab80681 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:51.745090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ed3c4-0ee2-492c-80bc-85c17ab80681 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:51.745480 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ed3c4-0ee2-492c-80bc-85c17ab80681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:51.770805 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-476ed3c4-0ee2-492c-80bc-85c17ab80681 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:51.772390 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-476ed3c4-0ee2-492c-80bc-85c17ab80681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:51.785110 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1260/2503] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:51.786301 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7c32e-cd45-4ff3-b786-4edad9b500b9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:51.797281 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7c32e-cd45-4ff3-b786-4edad9b500b9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:51.804771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:51.806028 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:51.807073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:51.807441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:51.809411 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:51.811576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:51.815101 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:51.832914 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1244/2504] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:51.834338 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ed3c4-0ee2-492c-80bc-85c17ab80681 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:51.845179 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ed3c4-0ee2-492c-80bc-85c17ab80681 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:51.853210 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:51.853858 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:51.854499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:51.854499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:51.855022 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:51.857003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:51.891452 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1261/2505] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:51 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:51.894294 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:51.914540 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4618b6-333a-4b89-b8c4-198dec1866b5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.015844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.016481 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.016710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.016975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.017602 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.019546 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.051542 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.055557 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1245/2506] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:51 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:52.064741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f73fd-88bb-43c2-a01c-f978173c793a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.071119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.071803 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.072154 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.072462 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.073369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.075429 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.086467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1262/2507] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:11:52.090485 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.101039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9144d3-8e71-4174-a5b7-80f18418189f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.105979 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57084236-412c-4227-acb8-c14f6bbd9dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.106525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57084236-412c-4227-acb8-c14f6bbd9dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.106778 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57084236-412c-4227-acb8-c14f6bbd9dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.107114 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57084236-412c-4227-acb8-c14f6bbd9dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.107876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57084236-412c-4227-acb8-c14f6bbd9dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.110148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57084236-412c-4227-acb8-c14f6bbd9dbb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.122329 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1246/2508] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:52.126367 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.136423 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4abe40-b018-44da-821b-b97ca49d9fd3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.142298 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.142944 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.143278 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.143561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] PATH_INFO: `/v3/projects/14d67732b864493ab1711b85c5aca214/users/806d3967144a4df6b6eb48ce82d5825a/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.174651 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d67732b864493ab1711b85c5aca214, user_id=806d3967144a4df6b6eb48ce82d5825a, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.178248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.196023 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1263/2509] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:52 2022] PUT /identity/v3/projects/14d67732b864493ab1711b85c5aca214/users/806d3967144a4df6b6eb48ce82d5825a/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:52.209459 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cbd32-3c03-454d-8ab4-9c753ed5724c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.209951 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cbd32-3c03-454d-8ab4-9c753ed5724c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.210307 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cbd32-3c03-454d-8ab4-9c753ed5724c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.242345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6a4cbd32-3c03-454d-8ab4-9c753ed5724c None None] MFA Rules not processed for user `806d3967144a4df6b6eb48ce82d5825a`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:52.243687 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a4cbd32-3c03-454d-8ab4-9c753ed5724c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:52.267052 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:52.325835 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1247/2510] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:52.330981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.416076 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbc83c-f221-4e0a-8d10-87181d773741 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.421403 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.422120 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.422653 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.422937 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.423630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.425559 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.460375 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1264/2511] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:52 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:52.464279 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cbd32-3c03-454d-8ab4-9c753ed5724c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.479907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cbd32-3c03-454d-8ab4-9c753ed5724c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.566610 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.567523 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.568026 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.568475 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.569601 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.571840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.601748 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1248/2512] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:52 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:52.607365 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.621476 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099b9a-b618-4938-aa05-e45933943375 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.628694 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.629379 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.629988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.629988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.630602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.632258 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.662514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1265/2513] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:52.666646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.679469 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c2efe-7f93-4e37-91f2-aa64e01d6699 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.684125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.684783 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.685056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.685326 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] PATH_INFO: `/v3/projects/19dc402afe1a4234b5d25b5febfa75a8/users/2c1b416808ac40e4bec867a5cad335c4/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.701968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19dc402afe1a4234b5d25b5febfa75a8, user_id=2c1b416808ac40e4bec867a5cad335c4, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.706318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.726241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1249/2514] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:52 2022] PUT /identity/v3/projects/19dc402afe1a4234b5d25b5febfa75a8/users/2c1b416808ac40e4bec867a5cad335c4/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:52.730256 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.740999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0e1bf-e2f0-4306-a7b0-c220c2aa047d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.777356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:52.814703 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca788c6-b688-42d1-a4eb-5e6b4d164324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.815432 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca788c6-b688-42d1-a4eb-5e6b4d164324 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.815711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca788c6-b688-42d1-a4eb-5e6b4d164324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.815983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca788c6-b688-42d1-a4eb-5e6b4d164324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.816649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca788c6-b688-42d1-a4eb-5e6b4d164324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.818291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca788c6-b688-42d1-a4eb-5e6b4d164324 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.831257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1266/2515] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:52.834202 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:52.846971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f951ec-3f7b-47cf-adbf-a0db9ce69015 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:52.852025 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:52.852693 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.853135 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.853414 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] PATH_INFO: `/v3/projects/19dc402afe1a4234b5d25b5febfa75a8/users/2c1b416808ac40e4bec867a5cad335c4/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.855431 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19dc402afe1a4234b5d25b5febfa75a8, user_id=2c1b416808ac40e4bec867a5cad335c4, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:52.858162 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:52.870039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1250/2516] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:52 2022] PUT /identity/v3/projects/19dc402afe1a4234b5d25b5febfa75a8/users/2c1b416808ac40e4bec867a5cad335c4/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:52.880594 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34119ede-501b-4d98-9eea-175d631a2bb4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:52.880747 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34119ede-501b-4d98-9eea-175d631a2bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:52.881024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34119ede-501b-4d98-9eea-175d631a2bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:52.916955 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-34119ede-501b-4d98-9eea-175d631a2bb4 None None] MFA Rules not processed for user `2c1b416808ac40e4bec867a5cad335c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:52.918245 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-34119ede-501b-4d98-9eea-175d631a2bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:53.037027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1267/2517] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:53.049577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.051918 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:53.197997 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.198282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.198688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.200990 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74bee4-98e2-4232-a918-a5e668a21754 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.206944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.208143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.208419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.208686 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.221188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.223345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.238360 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] MFA Rules not processed for user `2dda9b6c5edd4dec87e4d23ecfcea5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:53.239148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1251/2518] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:53.239767 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:53.302149 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.303974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:53.317309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ece736-6ac8-4158-ab95-027977d6a18d service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.321898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.322658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.323043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.323517 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.329509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1268/2519] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:11:53.333792 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.335186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:53.342252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:53.386037 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:53.402943 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.405319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.408314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c68d-6ebd-4105-b583-b76ebbb9df7e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.413138 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.413736 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.414008 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.414276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.414914 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.416894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.428242 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1252/2520] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:53.459287 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1269/2521] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:53 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:53.463120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.475268 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d1c7d-cf93-475d-af43-5c942a48d5b9 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.567710 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.568319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.568669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.568856 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.569498 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.571476 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.596314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1253/2522] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:53 2022] POST /identity/v3/users => generated 295 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:53.601445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.612059 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cfbe3-1d60-4a5a-88d8-f3f98512eade admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.617099 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.617896 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.618139 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.618431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.619041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.620649 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.631701 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1270/2523] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:53.635939 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.646572 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1066b-9306-4960-a5a7-11f8189c1134 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.653385 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.654128 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.654479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.654879 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] PATH_INFO: `/v3/projects/9bc7f1ee12744f5db14f38236b58b790/users/20c0a681cf024f55904c2a4ed862b2e6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.672837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc7f1ee12744f5db14f38236b58b790, user_id=20c0a681cf024f55904c2a4ed862b2e6, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.675836 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.693099 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1254/2524] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:53 2022] PUT /identity/v3/projects/9bc7f1ee12744f5db14f38236b58b790/users/20c0a681cf024f55904c2a4ed862b2e6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:53.698682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.710286 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff251a-d76a-42b6-93d6-cbb98426fd78 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.781070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.781688 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.781968 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.782340 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.782979 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.784621 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.797196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1271/2525] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:53.801395 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:53.813792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea86c-c130-4ddc-9ec4-a6621c456624 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:53.818961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca5953-4c9a-491c-92f0-4af7a6aeeb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:53.819658 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca5953-4c9a-491c-92f0-4af7a6aeeb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:53.819933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca5953-4c9a-491c-92f0-4af7a6aeeb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:53.820203 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca5953-4c9a-491c-92f0-4af7a6aeeb65 admin admin] PATH_INFO: `/v3/projects/9bc7f1ee12744f5db14f38236b58b790/users/20c0a681cf024f55904c2a4ed862b2e6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:53.822397 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca5953-4c9a-491c-92f0-4af7a6aeeb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc7f1ee12744f5db14f38236b58b790, user_id=20c0a681cf024f55904c2a4ed862b2e6, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:53.825632 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca5953-4c9a-491c-92f0-4af7a6aeeb65 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:53.837941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1255/2526] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:53 2022] PUT /identity/v3/projects/9bc7f1ee12744f5db14f38236b58b790/users/20c0a681cf024f55904c2a4ed862b2e6/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:11:57.297917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:57.310846 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c3762-83d4-49fc-b45b-f689f24549ba admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:57.397538 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:57.398199 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:57.398473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:57.402741 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] PATH_INFO: `/v3/users/806d3967144a4df6b6eb48ce82d5825a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:57.406397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806d3967144a4df6b6eb48ce82d5825a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:57.408412 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:57.494111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806d3967144a4df6b6eb48ce82d5825a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:57.494627 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806d3967144a4df6b6eb48ce82d5825a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:57.520253 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:57.524492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806d3967144a4df6b6eb48ce82d5825a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:57.526699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806d3967144a4df6b6eb48ce82d5825a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:57.567239 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806d3967144a4df6b6eb48ce82d5825a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:57.611276 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1272/2527] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:57 2022] DELETE /identity/v3/users/806d3967144a4df6b6eb48ce82d5825a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:57.626727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:57.627649 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:57.630944 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:57.659899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:57.663229 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:57.802525 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1256/2528] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:57.820652 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:57.822136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:57.948580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607e1b-73bc-493b-9e20-97302825d160 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:57.955296 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32cc3-3194-4786-b9d9-653a09ed3684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:57.955991 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32cc3-3194-4786-b9d9-653a09ed3684 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:57.956332 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32cc3-3194-4786-b9d9-653a09ed3684 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:57.956606 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32cc3-3194-4786-b9d9-653a09ed3684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:57.975755 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32cc3-3194-4786-b9d9-653a09ed3684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:57.977212 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32cc3-3194-4786-b9d9-653a09ed3684 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:58.004133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1273/2529] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:11:58.473805 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:58.475458 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:58.488811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3d7c-50ee-4284-9bd3-8bc7818ee729 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:58.496363 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:58.497408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:58.497794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:58.498971 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] PATH_INFO: `/v3/projects/14d67732b864493ab1711b85c5aca214` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:58.500378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d67732b864493ab1711b85c5aca214)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:58.502471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:58.506061 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:58.594022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d67732b864493ab1711b85c5aca214', 'request_id': 'req-7caaefa1-2c3e-4085-aef0-1705e540c93b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.601763 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d67732b864493ab1711b85c5aca214', 'request_id': 'req-7caaefa1-2c3e-4085-aef0-1705e540c93b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.611235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d67732b864493ab1711b85c5aca214', 'request_id': 'req-7caaefa1-2c3e-4085-aef0-1705e540c93b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.627966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764600d4-5958-4a03-883c-13c34c516615 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:58.628445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764600d4-5958-4a03-883c-13c34c516615 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:58.628565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764600d4-5958-4a03-883c-13c34c516615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:58.638941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] The token cache is being invalidate because project 14d67732b864493ab1711b85c5aca214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:58.639463 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.640940 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.642884 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1257/2530] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:58 2022] DELETE /identity/v3/projects/14d67732b864493ab1711b85c5aca214 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:58.646876 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:58.648702 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:58.664965 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-764600d4-5958-4a03-883c-13c34c516615 None None] MFA Rules not processed for user `20c0a681cf024f55904c2a4ed862b2e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:58.666237 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-764600d4-5958-4a03-883c-13c34c516615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:58.720899 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caaefa1-2c3e-4085-aef0-1705e540c93b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:58.726837 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:58.727738 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:58.728180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:58.728645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] PATH_INFO: `/v3/users/2c1b416808ac40e4bec867a5cad335c4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:58.729853 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1b416808ac40e4bec867a5cad335c4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:58.731906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:58.751722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1274/2531] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:58.768133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764600d4-5958-4a03-883c-13c34c516615 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:58.775163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-764600d4-5958-4a03-883c-13c34c516615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:58.810298 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1b416808ac40e4bec867a5cad335c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.810974 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1b416808ac40e4bec867a5cad335c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.818698 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1b416808ac40e4bec867a5cad335c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.826712 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1b416808ac40e4bec867a5cad335c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.827907 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1b416808ac40e4bec867a5cad335c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:58.845425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:58.846894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1258/2532] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:11:58 2022] DELETE /identity/v3/users/2c1b416808ac40e4bec867a5cad335c4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:58.878356 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764600d4-5958-4a03-883c-13c34c516615 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:58.960044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4b258-a1b5-4169-bcb7-922226c788eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:58.960684 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4b258-a1b5-4169-bcb7-922226c788eb service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:58.960964 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4b258-a1b5-4169-bcb7-922226c788eb service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:58.961624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4b258-a1b5-4169-bcb7-922226c788eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:58.964038 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-46b4b258-a1b5-4169-bcb7-922226c788eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:59.079036 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4b258-a1b5-4169-bcb7-922226c788eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:59.080709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4b258-a1b5-4169-bcb7-922226c788eb service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:59.093835 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1275/2533] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:11:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:11:59.232809 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:59.234330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:59.264070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:59.366928 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0df14d-225b-4cd3-b227-437fded220da admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:59.372827 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:59.373494 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.373775 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.374077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] PATH_INFO: `/v3/projects/19dc402afe1a4234b5d25b5febfa75a8` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.376548 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19dc402afe1a4234b5d25b5febfa75a8)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:59.378526 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:59.450580 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19dc402afe1a4234b5d25b5febfa75a8', 'request_id': 'req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:59.458142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19dc402afe1a4234b5d25b5febfa75a8', 'request_id': 'req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:59.471044 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19dc402afe1a4234b5d25b5febfa75a8', 'request_id': 'req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:59.495319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] The token cache is being invalidate because project 19dc402afe1a4234b5d25b5febfa75a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:11:59.495619 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:59.496621 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ad1a6e8-9f76-41f9-b6df-c48eb5004a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:11:59.498645 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1259/2534] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:11:59 2022] DELETE /identity/v3/projects/19dc402afe1a4234b5d25b5febfa75a8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:11:59.504804 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072341-5680-4cd6-8334-e649ee18a186 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.505395 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072341-5680-4cd6-8334-e649ee18a186 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.506348 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072341-5680-4cd6-8334-e649ee18a186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.548779 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c7072341-5680-4cd6-8334-e649ee18a186 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:59.550018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c7072341-5680-4cd6-8334-e649ee18a186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:59.644362 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1276/2535] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:59.652844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959cad9-5f32-4066-8e7d-6d168d43ad47 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.653825 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959cad9-5f32-4066-8e7d-6d168d43ad47 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.654981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959cad9-5f32-4066-8e7d-6d168d43ad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.678530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a959cad9-5f32-4066-8e7d-6d168d43ad47 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:11:59.679908 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a959cad9-5f32-4066-8e7d-6d168d43ad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:11:59.693232 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1260/2536] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:11:59.703051 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7072341-5680-4cd6-8334-e649ee18a186 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:59.720391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7072341-5680-4cd6-8334-e649ee18a186 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:59.734000 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:59.734704 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.734988 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.735282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.736104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:11:59.736577 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:59.738368 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:59.754170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1277/2537] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:11:59.758975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959cad9-5f32-4066-8e7d-6d168d43ad47 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:59.770457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959cad9-5f32-4066-8e7d-6d168d43ad47 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:59.775147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:59.775774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.776057 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.776324 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.777117 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:59.779029 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:59.808870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1261/2538] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:11:59 2022] POST /identity/v3/projects => generated 401 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:11:59.813807 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:59.826087 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81860a-3d4c-423c-866f-309f0442f25a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:59.846634 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:11:59.887976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:59.888582 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.888865 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.889136 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.890334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:59.892295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:59.913570 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1278/2539] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:11:59 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:11:59.918473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:59.930219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a04d1-c0a8-4b0d-850e-aa917a3a2bda admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:59.937194 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:59.937997 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.938495 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.938868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.940382 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:59.942292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:59.955664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1262/2540] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:59.960257 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:11:59.972064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8d1c-63a4-49ed-840f-80a8dc809037 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:11:59.976573 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771b5d6-e68e-4732-8093-c48ec21988e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:11:59.977171 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771b5d6-e68e-4732-8093-c48ec21988e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:11:59.977445 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771b5d6-e68e-4732-8093-c48ec21988e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:11:59.977740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771b5d6-e68e-4732-8093-c48ec21988e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:11:59.978391 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771b5d6-e68e-4732-8093-c48ec21988e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:11:59.980010 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771b5d6-e68e-4732-8093-c48ec21988e6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:11:59.991193 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1279/2541] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:11:59.995504 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.006585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db46b9-add9-4fec-812b-b483e54b1797 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.011231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.011893 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.012169 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.012441 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] PATH_INFO: `/v3/projects/3f3446d8704d4a929d23a4d35e52f07c/users/95f1b0f15cce49cab4bfbffcd1df7510/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.027931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3446d8704d4a929d23a4d35e52f07c, user_id=95f1b0f15cce49cab4bfbffcd1df7510, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.030615 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.050186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1263/2542] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:11:59 2022] PUT /identity/v3/projects/3f3446d8704d4a929d23a4d35e52f07c/users/95f1b0f15cce49cab4bfbffcd1df7510/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:00.123444 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4f9ba-cd29-4557-9676-a6a7f7ec79a7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.123707 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4f9ba-cd29-4557-9676-a6a7f7ec79a7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.123946 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4f9ba-cd29-4557-9676-a6a7f7ec79a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.154277 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7fb4f9ba-cd29-4557-9676-a6a7f7ec79a7 None None] MFA Rules not processed for user `95f1b0f15cce49cab4bfbffcd1df7510`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:00.157325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7fb4f9ba-cd29-4557-9676-a6a7f7ec79a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:00.241724 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1280/2543] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:00.246524 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.298665 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:00.324124 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbaa5b-3189-48dc-842f-6b85e467aacc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.329705 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.330309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.330587 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.330855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.331520 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.333448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.369110 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1264/2544] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:12:00 2022] POST /identity/v3/projects => generated 401 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:12:00.370152 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4f9ba-cd29-4557-9676-a6a7f7ec79a7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.381383 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4f9ba-cd29-4557-9676-a6a7f7ec79a7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.440035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.440678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.440960 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.441232 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.441869 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.443848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.471323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1281/2545] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:12:00 2022] POST /identity/v3/users => generated 307 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:12:00.473564 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.484631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c3f5c-b84f-4eba-be2f-e49dee71f082 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.489305 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.489905 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.490180 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.490451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.491073 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.493978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.505260 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1265/2546] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:12:00.510260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.522208 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb95471-0dcc-423e-8941-0c88a288a735 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.527509 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.528170 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.528453 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.528734 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] PATH_INFO: `/v3/projects/387c3b5ffa2241a785e1f7fa69d9cf0e/users/3bb8f52e45784e8282680fecd3d9b541/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.546722 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387c3b5ffa2241a785e1f7fa69d9cf0e, user_id=3bb8f52e45784e8282680fecd3d9b541, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.549153 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.552828 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:00.570111 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1282/2547] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:12:00 2022] PUT /identity/v3/projects/387c3b5ffa2241a785e1f7fa69d9cf0e/users/3bb8f52e45784e8282680fecd3d9b541/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:12:00.571361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.585277 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0ecc2-e5e1-49ea-9250-b04a297d2e26 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.652109 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30043-4de6-4f91-ac72-38fb2c79a6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.652727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30043-4de6-4f91-ac72-38fb2c79a6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.653004 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30043-4de6-4f91-ac72-38fb2c79a6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.653901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30043-4de6-4f91-ac72-38fb2c79a6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.654915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c30043-4de6-4f91-ac72-38fb2c79a6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.656941 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c30043-4de6-4f91-ac72-38fb2c79a6f3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.668039 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1266/2548] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:12:00.672902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.684325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31369f-6545-4fc9-9004-65b9b347f8e6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.691030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.692149 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.692553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.692983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] PATH_INFO: `/v3/projects/387c3b5ffa2241a785e1f7fa69d9cf0e/users/3bb8f52e45784e8282680fecd3d9b541/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.695248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387c3b5ffa2241a785e1f7fa69d9cf0e, user_id=3bb8f52e45784e8282680fecd3d9b541, role_id=cd014fd2ff9b4b0ea2f79f9f55c087b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.698243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.710352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1283/2549] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:12:00 2022] PUT /identity/v3/projects/387c3b5ffa2241a785e1f7fa69d9cf0e/users/3bb8f52e45784e8282680fecd3d9b541/roles/cd014fd2ff9b4b0ea2f79f9f55c087b9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:12:00.721840 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afdf0c-16ad-4d1d-b47a-55ed7dcebdca None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.722318 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afdf0c-16ad-4d1d-b47a-55ed7dcebdca None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.722752 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afdf0c-16ad-4d1d-b47a-55ed7dcebdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.765538 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f9afdf0c-16ad-4d1d-b47a-55ed7dcebdca None None] MFA Rules not processed for user `3bb8f52e45784e8282680fecd3d9b541`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:00.767088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f9afdf0c-16ad-4d1d-b47a-55ed7dcebdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:00.846051 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1267/2550] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:00 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:12:00.858293 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:00.861026 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:00.919554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d614c8-6f3c-4c65-b93a-4736331641b2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:00.923288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:00.923884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:00.924161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:00.924433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:00.934260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:00.935928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:00.946994 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1284/2551] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:00 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:01.054347 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afdf0c-16ad-4d1d-b47a-55ed7dcebdca None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:01.066120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afdf0c-16ad-4d1d-b47a-55ed7dcebdca None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:01.070381 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5e268-9f0e-41f9-b62c-2011212ec551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:01.071070 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5e268-9f0e-41f9-b62c-2011212ec551 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:01.071457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5e268-9f0e-41f9-b62c-2011212ec551 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:01.071802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5e268-9f0e-41f9-b62c-2011212ec551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:01.109915 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:01.141244 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5e268-9f0e-41f9-b62c-2011212ec551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:01.142704 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5e268-9f0e-41f9-b62c-2011212ec551 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:01.154185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1268/2552] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:01 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:05.354249 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:05.416976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99267bdf-8a25-40ed-851e-700a1f3f8cd7 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:05.422246 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:05.422886 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:05.423330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:05.423635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] PATH_INFO: `/v3/users/95f1b0f15cce49cab4bfbffcd1df7510` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:05.424900 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f1b0f15cce49cab4bfbffcd1df7510)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:05.426891 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:05.474739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f1b0f15cce49cab4bfbffcd1df7510'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:05.475402 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f1b0f15cce49cab4bfbffcd1df7510'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:05.482243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f1b0f15cce49cab4bfbffcd1df7510'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:05.482875 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f1b0f15cce49cab4bfbffcd1df7510'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:05.495027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f1b0f15cce49cab4bfbffcd1df7510'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:05.507823 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1285/2553] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:12:05 2022] DELETE /identity/v3/users/95f1b0f15cce49cab4bfbffcd1df7510 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:05.512985 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:05.513868 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:05.514142 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:05.532967 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:05.534090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:05.593749 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1269/2554] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:05.603714 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:05.605150 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:05.612260 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:05.665549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12162d61-69fb-4ffe-835d-53b9399af5c3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:05.669639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:05.670345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:05.670676 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:05.671074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:05.680030 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:05.681481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:05.690973 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1286/2555] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:05.973577 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:05.975223 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:05.987384 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6f59e-2cc5-48a0-b915-01ec9b7beceb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:05.991942 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:05.992573 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:05.992851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:05.993120 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] PATH_INFO: `/v3/projects/3f3446d8704d4a929d23a4d35e52f07c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:05.994231 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3446d8704d4a929d23a4d35e52f07c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:05.996192 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:06.045680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3446d8704d4a929d23a4d35e52f07c', 'request_id': 'req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.051727 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3446d8704d4a929d23a4d35e52f07c', 'request_id': 'req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.062270 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3446d8704d4a929d23a4d35e52f07c', 'request_id': 'req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.074034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] The token cache is being invalidate because project 3f3446d8704d4a929d23a4d35e52f07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:12:06.074317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.075198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.079290 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1270/2556] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:12:05 2022] DELETE /identity/v3/projects/3f3446d8704d4a929d23a4d35e52f07c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:06.081459 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:06.090897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e67e-9eb0-4b4c-93fe-00d912f16962 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:06.150689 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:06.151366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:06.151643 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:06.151911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] PATH_INFO: `/v3/users/3bb8f52e45784e8282680fecd3d9b541` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:06.153040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb8f52e45784e8282680fecd3d9b541)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:06.155339 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:06.206531 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb8f52e45784e8282680fecd3d9b541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.207163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb8f52e45784e8282680fecd3d9b541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.213909 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb8f52e45784e8282680fecd3d9b541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.214529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb8f52e45784e8282680fecd3d9b541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.228128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0368d654-f13b-4eb7-a897-608f2340a5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb8f52e45784e8282680fecd3d9b541'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.242844 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1287/2557] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:12:06 2022] DELETE /identity/v3/users/3bb8f52e45784e8282680fecd3d9b541 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:12:06.526966 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:06.528440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:06.543461 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b984c-8ed3-492c-bfa7-437c7f5dc0c3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:06.603387 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:06.613008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:06.613728 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:06.614178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:06.614600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] PATH_INFO: `/v3/projects/387c3b5ffa2241a785e1f7fa69d9cf0e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:06.615838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387c3b5ffa2241a785e1f7fa69d9cf0e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:06.617731 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:06.665401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387c3b5ffa2241a785e1f7fa69d9cf0e', 'request_id': 'req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.673843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387c3b5ffa2241a785e1f7fa69d9cf0e', 'request_id': 'req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.682791 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387c3b5ffa2241a785e1f7fa69d9cf0e', 'request_id': 'req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.694343 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] The token cache is being invalidate because project 387c3b5ffa2241a785e1f7fa69d9cf0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:12:06.694631 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.695293 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2f70e669-9a1b-4c39-a4ef-3c005af0e0d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:06.696781 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1271/2558] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:12:06 2022] DELETE /identity/v3/projects/387c3b5ffa2241a785e1f7fa69d9cf0e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:06.702035 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6843e-310b-4913-8058-88825ee3894a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:06.702507 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6843e-310b-4913-8058-88825ee3894a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:06.702608 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6843e-310b-4913-8058-88825ee3894a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:06.722852 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2ba6843e-310b-4913-8058-88825ee3894a None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:06.724106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2ba6843e-310b-4913-8058-88825ee3894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:06.745064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:06.791106 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1288/2559] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:06.799652 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afae0f-7a0e-499b-8c8c-45ff046d5d73 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:06.799948 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afae0f-7a0e-499b-8c8c-45ff046d5d73 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:06.800252 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afae0f-7a0e-499b-8c8c-45ff046d5d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:06.820095 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-71afae0f-7a0e-499b-8c8c-45ff046d5d73 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:06.821339 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-71afae0f-7a0e-499b-8c8c-45ff046d5d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:06.829994 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1272/2560] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:06.834217 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6843e-310b-4913-8058-88825ee3894a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:06.844708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6843e-310b-4913-8058-88825ee3894a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:06.849460 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:06.850063 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:06.850336 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:06.850602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:06.851282 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:12:06.851581 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:06.853168 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:06.865388 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1289/2561] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:12:06.869864 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afae0f-7a0e-499b-8c8c-45ff046d5d73 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:06.879081 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afae0f-7a0e-499b-8c8c-45ff046d5d73 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:06.884125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:06.884745 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:06.885056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:06.885327 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:06.885968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:06.887931 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:06.915984 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1273/2562] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:12:06 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:12:06.920189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:06.935157 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd714c-c1d0-41b7-8d42-862ec136a030 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:06.996467 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:06.997070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:06.997352 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:06.997624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:06.998255 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:07.000203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:07.022435 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1290/2563] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:12:06 2022] POST /identity/v3/users => generated 280 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 07:12:07.026583 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:07.037576 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e7f5f-cdc1-4a4b-82c8-8d16eb7f2f78 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:07.042236 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:07.042828 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.043105 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.043530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.044164 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:07.045755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:07.055801 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1274/2564] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:12:07.060070 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:07.073434 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80991fe7-96f8-41fc-9259-ebcc705f34bc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:07.078830 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fe7d7-64a9-4775-9942-913dde1dfd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:07.079454 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe7d7-64a9-4775-9942-913dde1dfd66 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.079729 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe7d7-64a9-4775-9942-913dde1dfd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.080284 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe7d7-64a9-4775-9942-913dde1dfd66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.080908 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fe7d7-64a9-4775-9942-913dde1dfd66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:07.082489 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fe7d7-64a9-4775-9942-913dde1dfd66 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:07.093094 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1291/2565] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:12:07.097603 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:07.111996 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30ee8d-e941-4c1a-965f-21c4cabff8cb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:07.116973 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:07.117633 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.117913 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.118188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] PATH_INFO: `/v3/projects/f39c71a256c84447943eaf1c892c6206/users/5f9c4158b4364d6eb1b95e93feb93fdd/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.133015 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39c71a256c84447943eaf1c892c6206, user_id=5f9c4158b4364d6eb1b95e93feb93fdd, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:07.135751 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:07.151991 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1275/2566] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:12:07 2022] PUT /identity/v3/projects/f39c71a256c84447943eaf1c892c6206/users/5f9c4158b4364d6eb1b95e93feb93fdd/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:07.164330 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a34a-5183-4030-8f50-191fb0dbe743 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.164630 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a34a-5183-4030-8f50-191fb0dbe743 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.164902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a34a-5183-4030-8f50-191fb0dbe743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.192077 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-87f0a34a-5183-4030-8f50-191fb0dbe743 None None] MFA Rules not processed for user `5f9c4158b4364d6eb1b95e93feb93fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:07.193836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-87f0a34a-5183-4030-8f50-191fb0dbe743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:07.261431 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1292/2567] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:07.295779 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:07.297570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:07.355512 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af63c9-fbad-4b4f-83f3-e0e8490d519d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:07.359199 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dd348-6a9c-4a3e-9c24-33b36cc0a454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:07.359851 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dd348-6a9c-4a3e-9c24-33b36cc0a454 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.360130 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dd348-6a9c-4a3e-9c24-33b36cc0a454 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.360401 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dd348-6a9c-4a3e-9c24-33b36cc0a454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.369378 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dd348-6a9c-4a3e-9c24-33b36cc0a454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:07.370750 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dd348-6a9c-4a3e-9c24-33b36cc0a454 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:07.380680 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1276/2568] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:07.397024 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0a34a-5183-4030-8f50-191fb0dbe743 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:07.422518 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:07.462359 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0a34a-5183-4030-8f50-191fb0dbe743 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:07.466905 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:07.467590 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.467872 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.468145 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] PATH_INFO: `/v3/users/5f9c4158b4364d6eb1b95e93feb93fdd` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.469270 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f9c4158b4364d6eb1b95e93feb93fdd)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:07.471189 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:07.518733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f9c4158b4364d6eb1b95e93feb93fdd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:07.519417 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f9c4158b4364d6eb1b95e93feb93fdd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:07.526173 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f9c4158b4364d6eb1b95e93feb93fdd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:07.526822 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f9c4158b4364d6eb1b95e93feb93fdd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:07.538526 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f9c4158b4364d6eb1b95e93feb93fdd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:07.552585 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1293/2569] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:12:07 2022] DELETE /identity/v3/users/5f9c4158b4364d6eb1b95e93feb93fdd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:07.556972 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.557457 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.557558 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.577834 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:07.579620 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:07.598724 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:07.643789 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1277/2570] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:07.653535 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:07.654918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:07.713039 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560640-c0bc-4564-a12b-b128bc6b7b52 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:07.716709 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc55717-b739-4239-9075-59869d709660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:07.717314 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc55717-b739-4239-9075-59869d709660 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:07.717593 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc55717-b739-4239-9075-59869d709660 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:07.717862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc55717-b739-4239-9075-59869d709660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:07.727108 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc55717-b739-4239-9075-59869d709660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:07.728553 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc55717-b739-4239-9075-59869d709660 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:07.738411 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1294/2571] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:08.013442 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.014894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:08.027166 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b96a8-fda1-47ce-a2fe-34d3a8437a82 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.032992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.033726 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.034001 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.034664 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] PATH_INFO: `/v3/projects/f39c71a256c84447943eaf1c892c6206` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.036747 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f39c71a256c84447943eaf1c892c6206)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.039549 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.088002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f39c71a256c84447943eaf1c892c6206', 'request_id': 'req-d2009000-f63c-441a-a33f-554780eb38a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:08.093559 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f39c71a256c84447943eaf1c892c6206', 'request_id': 'req-d2009000-f63c-441a-a33f-554780eb38a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:08.101511 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f39c71a256c84447943eaf1c892c6206', 'request_id': 'req-d2009000-f63c-441a-a33f-554780eb38a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:08.113188 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] The token cache is being invalidate because project f39c71a256c84447943eaf1c892c6206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:12:08.113467 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:08.114074 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d2009000-f63c-441a-a33f-554780eb38a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:08.115579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1278/2572] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:12:08 2022] DELETE /identity/v3/projects/f39c71a256c84447943eaf1c892c6206 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:08.120599 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0925f-e5e6-4545-982f-86296a775bf3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.120885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0925f-e5e6-4545-982f-86296a775bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.121161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0925f-e5e6-4545-982f-86296a775bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.141498 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-76a0925f-e5e6-4545-982f-86296a775bf3 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:08.142638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-76a0925f-e5e6-4545-982f-86296a775bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:08.199780 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:08.208918 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1295/2573] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:08.216760 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e1f2-7a0b-460b-be66-04414d38e2ab None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.217043 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e1f2-7a0b-460b-be66-04414d38e2ab None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.217345 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e1f2-7a0b-460b-be66-04414d38e2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.238437 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4d76e1f2-7a0b-460b-be66-04414d38e2ab None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:08.239733 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4d76e1f2-7a0b-460b-be66-04414d38e2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:08.247600 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1279/2574] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:08.251981 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0925f-e5e6-4545-982f-86296a775bf3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.261904 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0925f-e5e6-4545-982f-86296a775bf3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.266622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.267329 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.267613 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.267885 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.268529 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 07:12:08.268836 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.270433 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.282820 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1296/2575] 10.222.0.36 () {58 vars in 1153 bytes} [Mon Oct 17 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 07:12:08.287176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e1f2-7a0b-460b-be66-04414d38e2ab None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.296518 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e1f2-7a0b-460b-be66-04414d38e2ab None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.302033 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.302670 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.302968 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.303313 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.303960 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.305898 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.333992 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1280/2576] 10.222.0.36 () {60 vars in 1153 bytes} [Mon Oct 17 07:12:08 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:12:08.338073 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.350587 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac1000-9a7d-4dce-a5d4-f0abaf811bcf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.407018 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.407667 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.407949 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.408219 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.408837 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.410740 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.435288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1297/2577] 10.222.0.36 () {60 vars in 1144 bytes} [Mon Oct 17 07:12:08 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 07:12:08.436510 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.447803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce100a-c8aa-4520-9b97-9bd635f9b3f6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.452794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.453875 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.454145 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.454408 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.455027 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.457185 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.466757 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1281/2578] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 07:12:08.472295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.484275 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426de84-b5bf-4242-94ec-7a03066c30cc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.488932 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcf190-c178-4b87-8747-9cbf2016b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.489521 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcf190-c178-4b87-8747-9cbf2016b167 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.489795 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcf190-c178-4b87-8747-9cbf2016b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.490064 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcf190-c178-4b87-8747-9cbf2016b167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.490686 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcf190-c178-4b87-8747-9cbf2016b167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.492297 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcf190-c178-4b87-8747-9cbf2016b167 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.502728 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1298/2579] 10.222.0.36 () {58 vars in 1122 bytes} [Mon Oct 17 07:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 07:12:08.507035 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.517881 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171baebb-ac6a-48f1-bf6f-fd5bbf5ac08f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.522870 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.523566 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.523849 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.524125 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] PATH_INFO: `/v3/projects/9ae8fb329c9a4e5a950f695da4dbe16f/users/5fb9a6fa97ee4131a31acb672c5e7d59/roles/84eb2fe76ba84dfab84d2d1ab664754f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.539648 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae8fb329c9a4e5a950f695da4dbe16f, user_id=5fb9a6fa97ee4131a31acb672c5e7d59, role_id=84eb2fe76ba84dfab84d2d1ab664754f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.542353 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.554958 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1282/2580] 10.222.0.36 () {60 vars in 1483 bytes} [Mon Oct 17 07:12:08 2022] PUT /identity/v3/projects/9ae8fb329c9a4e5a950f695da4dbe16f/users/5fb9a6fa97ee4131a31acb672c5e7d59/roles/84eb2fe76ba84dfab84d2d1ab664754f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:08.566884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeedfa8-1376-4995-8974-d4df244b4e62 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.567166 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeedfa8-1376-4995-8974-d4df244b4e62 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.567494 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeedfa8-1376-4995-8974-d4df244b4e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.602369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8aeedfa8-1376-4995-8974-d4df244b4e62 None None] MFA Rules not processed for user `5fb9a6fa97ee4131a31acb672c5e7d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:08.603616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8aeedfa8-1376-4995-8974-d4df244b4e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:08.671716 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1299/2581] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:08.683361 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:08.685322 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:08.732198 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:08.744330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb49d1a-1c07-4193-9097-f8377c568043 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:08.748065 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c66f0a106f4c8e99ac7075d76767d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:08.748659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:08.748933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:08.749200 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:08.757839 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:08.759314 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:08.768508 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1283/2582] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:18.668713 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:18.763894 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:20.931514 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab585c-3307-4b1f-9100-ea3ff96eb1ed None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:20.931976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab585c-3307-4b1f-9100-ea3ff96eb1ed None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:20.932078 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab585c-3307-4b1f-9100-ea3ff96eb1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:20.951483 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-23ab585c-3307-4b1f-9100-ea3ff96eb1ed None None] MFA Rules not processed for user `2dda9b6c5edd4dec87e4d23ecfcea5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:20.952853 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-23ab585c-3307-4b1f-9100-ea3ff96eb1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:21.006371 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1300/2583] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:28.450625 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:28.511006 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22039fc8-d9ef-444b-b093-495e1c0bac44 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:28.515440 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:28.516048 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:28.516319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:28.516588 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] PATH_INFO: `/v3/users/5fb9a6fa97ee4131a31acb672c5e7d59` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:28.517669 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb9a6fa97ee4131a31acb672c5e7d59)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:28.520140 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:28.563812 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb9a6fa97ee4131a31acb672c5e7d59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:28.564386 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb9a6fa97ee4131a31acb672c5e7d59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:28.572291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb9a6fa97ee4131a31acb672c5e7d59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:28.580586 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb9a6fa97ee4131a31acb672c5e7d59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:28.581818 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb9a6fa97ee4131a31acb672c5e7d59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:28.598983 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1284/2584] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:12:28 2022] DELETE /identity/v3/users/5fb9a6fa97ee4131a31acb672c5e7d59 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:12:28.603565 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:28.604044 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:28.604163 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:28.623325 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] MFA Rules not processed for user `be01c4fc32bc463e9e081f13d9758f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:12:28.624517 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:28.682248 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1301/2585] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:12:28.692961 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:28.694439 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:28.752195 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bc722-a55b-4b32-9275-f918d8bb16ac admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:28.756372 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:28.756978 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:28.757253 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:28.757530 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:28.766765 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:28.768817 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:28.778416 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1285/2586] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:29.323234 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:29.324698 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:29.336863 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bea06-b6c3-4002-89e3-6bc656daa977 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:29.341928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:29.342637 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:29.342983 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:29.343366 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] PATH_INFO: `/v3/projects/9ae8fb329c9a4e5a950f695da4dbe16f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:29.345369 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae8fb329c9a4e5a950f695da4dbe16f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:29.347808 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:29.403963 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae8fb329c9a4e5a950f695da4dbe16f', 'request_id': 'req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:29.410118 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae8fb329c9a4e5a950f695da4dbe16f', 'request_id': 'req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:29.416487 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae8fb329c9a4e5a950f695da4dbe16f', 'request_id': 'req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:29.429006 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] The token cache is being invalidate because project 9ae8fb329c9a4e5a950f695da4dbe16f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:12:29.429288 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:29.429902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:12:29.431425 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1302/2587] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:12:29 2022] DELETE /identity/v3/projects/9ae8fb329c9a4e5a950f695da4dbe16f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:12:38.774090 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:39.425976 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:39.633285 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:39.641824 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450314b4-ee23-4dca-8e3e-c325bf33a1d2 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:39.689319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:39.689933 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:39.690210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:39.690493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:39.692251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:39.754369 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:39.755830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:39.765176 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1286/2588] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:49.760892 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=50 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:12:56.832595 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:56.834041 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:56.845928 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73da60-cec1-4a2b-98b9-d6c8c4cf49f7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:56.850090 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:56.850841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:56.851280 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:56.851560 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:56.853337 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:56.924119 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:56.926075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:56.937104 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1303/2589] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:12:58.117949 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:12:58.126579 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e3e51-bc18-4ea6-bd7d-01db6af7a9d0 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:12:58.130771 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:12:58.131392 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:12:58.131689 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:12:58.131955 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:12:58.133627 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:12:58.147838 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:12:58.149280 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:12:58.161241 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1287/2590] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:12:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:13:06.932115 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=34 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:13:08.155471 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:14:15.447449 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:15.456404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfacab-1383-4cd8-b2b8-befc2750337f service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:15.459982 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:15.460561 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:15.460834 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:15.461123 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:15.462812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:15.472616 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:15.474014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:15.482499 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1304/2591] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:14:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:14:18.228086 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:18.230493 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:18.246697 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02e122-d8d3-45ce-9152-b1bb78b55065 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:18.252210 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:18.252811 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:18.253088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:18.253357 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:18.255150 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:18.267482 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:18.269079 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:18.280901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1288/2592] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:14:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:14:23.299897 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:23.301321 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:23.311555 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7913ba-28df-45bf-aa6b-7d5aaea483a0 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:23.315068 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:23.315683 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:23.315966 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:23.316221 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:23.317841 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:23.379158 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:23.380603 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:23.390711 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1305/2593] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:14:26.492596 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:26.494088 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:26.580594 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861dea-abc5-4c12-8a12-3cafec782972 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:26.585766 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:26.586412 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:26.586691 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:26.586963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:26.588906 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:26.602304 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:26.604330 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:26.615774 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1289/2594] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:14:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:14:29.419027 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:29.420554 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:29.437838 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee684bf2-8fdf-466f-8e2a-f1db65d67df3 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:29.444611 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:29.445881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:29.445881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:29.445881 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] PATH_INFO: `/v3/users/455a8022fa3543b2a0ce0cdfc1512f73` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:29.447057 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=455a8022fa3543b2a0ce0cdfc1512f73)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:29.450680 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:29.508504 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455a8022fa3543b2a0ce0cdfc1512f73'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.509144 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455a8022fa3543b2a0ce0cdfc1512f73'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.516974 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455a8022fa3543b2a0ce0cdfc1512f73'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.517609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455a8022fa3543b2a0ce0cdfc1512f73'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.529397 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455a8022fa3543b2a0ce0cdfc1512f73'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.543101 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1306/2595] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:14:29 2022] DELETE /identity/v3/users/455a8022fa3543b2a0ce0cdfc1512f73 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:14:29.724993 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:29.736628 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b1c2b-782a-4580-8d0e-6b9294010f27 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:29.795570 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:29.796193 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:29.796473 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:29.796742 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] PATH_INFO: `/v3/projects/3326642f5c9e4103aa3c72f47b5d23b5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:29.797826 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3326642f5c9e4103aa3c72f47b5d23b5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:29.799785 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:29.845479 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3326642f5c9e4103aa3c72f47b5d23b5', 'request_id': 'req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.851077 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3326642f5c9e4103aa3c72f47b5d23b5', 'request_id': 'req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.858407 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3326642f5c9e4103aa3c72f47b5d23b5', 'request_id': 'req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.869434 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] The token cache is being invalidate because project 3326642f5c9e4103aa3c72f47b5d23b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:14:29.869717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.870975 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.872470 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1290/2596] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:14:29 2022] DELETE /identity/v3/projects/3326642f5c9e4103aa3c72f47b5d23b5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:14:29.876497 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:29.877902 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:29.890901 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c668e-c3c2-4d7d-9f3e-ce66e3530c96 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:29.937812 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:14:29.947644 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:29.948265 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:29.948580 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:29.948856 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] PATH_INFO: `/v3/users/389ac9a127eb471884975a97fb5ee026` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:29.950243 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=389ac9a127eb471884975a97fb5ee026)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:29.952186 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:29.997682 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389ac9a127eb471884975a97fb5ee026'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:29.998308 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389ac9a127eb471884975a97fb5ee026'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.005083 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389ac9a127eb471884975a97fb5ee026'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.005699 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389ac9a127eb471884975a97fb5ee026'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.017185 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3537510-b53e-4ae9-b17d-8b7212538200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389ac9a127eb471884975a97fb5ee026'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.029639 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1307/2597] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:14:29 2022] DELETE /identity/v3/users/389ac9a127eb471884975a97fb5ee026 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:14:30.206598 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:30.208041 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:30.223802 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48a2ae-1ee1-473b-9fd2-323fde89cf25 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:30.228008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:14:30.283953 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:30.284571 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:30.284844 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:30.285112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] PATH_INFO: `/v3/projects/408f1ef15360431c92948d09ad9b7336` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:30.286212 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=408f1ef15360431c92948d09ad9b7336)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:30.288178 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:30.332855 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '408f1ef15360431c92948d09ad9b7336', 'request_id': 'req-207b7192-e2d0-4888-aaa6-28f5560485a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.338179 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '408f1ef15360431c92948d09ad9b7336', 'request_id': 'req-207b7192-e2d0-4888-aaa6-28f5560485a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.346008 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '408f1ef15360431c92948d09ad9b7336', 'request_id': 'req-207b7192-e2d0-4888-aaa6-28f5560485a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.357723 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] The token cache is being invalidate because project 408f1ef15360431c92948d09ad9b7336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:14:30.358002 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.358671 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:14:30.360147 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1291/2598] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:14:30 2022] DELETE /identity/v3/projects/408f1ef15360431c92948d09ad9b7336 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:14:40.026128 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:14:40.354143 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:14:40.393151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafcbd7-e5fe-4593-80ae-127c201854e4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:40.393502 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafcbd7-e5fe-4593-80ae-127c201854e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:40.393843 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafcbd7-e5fe-4593-80ae-127c201854e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:40.414011 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0bafcbd7-e5fe-4593-80ae-127c201854e4 None None] MFA Rules not processed for user `2dda9b6c5edd4dec87e4d23ecfcea5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:14:40.415334 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0bafcbd7-e5fe-4593-80ae-127c201854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:40.472291 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1308/2599] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:14:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:14:40.786449 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:40.787858 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:40.843719 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b7192-e2d0-4888-aaa6-28f5560485a7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:40.847278 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa418df7-70d7-4588-9148-1d28f27af5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:40.847871 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa418df7-70d7-4588-9148-1d28f27af5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:40.848148 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa418df7-70d7-4588-9148-1d28f27af5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:40.848418 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa418df7-70d7-4588-9148-1d28f27af5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:40.850083 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aa418df7-70d7-4588-9148-1d28f27af5a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:40.860409 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa418df7-70d7-4588-9148-1d28f27af5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:40.861830 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa418df7-70d7-4588-9148-1d28f27af5a4 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:40.869668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1292/2600] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:14:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:14:41.314295 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745740f-9eea-42e5-9200-646fb16f0ea9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:41.314733 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745740f-9eea-42e5-9200-646fb16f0ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:41.314862 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745740f-9eea-42e5-9200-646fb16f0ea9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:41.316161 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1309/2601] 10.222.0.36 () {54 vars in 961 bytes} [Mon Oct 17 07:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 07:14:41.321500 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:41.321812 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:41.322132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:41.370695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] MFA Rules not processed for user `522ff98d9ecb4d90bd97a19f71a400c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:14:41.371963 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:41.433072 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1293/2602] 10.222.0.36 () {58 vars in 1058 bytes} [Mon Oct 17 07:14:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 07:14:41.441093 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745740f-9eea-42e5-9200-646fb16f0ea9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:41.454899 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745740f-9eea-42e5-9200-646fb16f0ea9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:41.459749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:41.460323 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:41.460598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:41.460859 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:41.462549 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:41.473540 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:41.475069 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:41.484671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1310/2603] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:14:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:14:51.424803 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=65 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:14:51.480447 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=31 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:14:53.425181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:14:53.426604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:53.501172 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dfcda-f6e4-4648-941b-2502b62c6319 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:14:53.506484 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a090c-39f7-4910-8be4-d66954ae88a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:14:53.507084 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a090c-39f7-4910-8be4-d66954ae88a7 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:14:53.507391 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a090c-39f7-4910-8be4-d66954ae88a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:14:53.507659 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a090c-39f7-4910-8be4-d66954ae88a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:14:53.509340 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0f1a090c-39f7-4910-8be4-d66954ae88a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:14:53.521667 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a090c-39f7-4910-8be4-d66954ae88a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:14:53.523639 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a090c-39f7-4910-8be4-d66954ae88a7 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:14:53.534181 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1294/2604] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:14:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 07:15:03.529591 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=34 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:16:18.740710 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:16:18.742089 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:16:18.754598 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494e60-7305-4771-a707-916e3c0354a1 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:16:18.759751 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:16:18.760342 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:16:18.760620 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:16:18.760917 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:16:18.762602 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:16:18.773739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:16:18.775335 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:16:18.785622 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1311/2605] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:16:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:16:28.780842 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:16:33.667341 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573563b2-6579-4032-9234-9c036f735e16 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:16:33.667843 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573563b2-6579-4032-9234-9c036f735e16 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:16:33.667957 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573563b2-6579-4032-9234-9c036f735e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:16:33.687251 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-573563b2-6579-4032-9234-9c036f735e16 None None] MFA Rules not processed for user `2dda9b6c5edd4dec87e4d23ecfcea5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 07:16:33.688451 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-573563b2-6579-4032-9234-9c036f735e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:16:33.695056 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1295/2606] 10.222.0.36 () {58 vars in 958 bytes} [Mon Oct 17 07:16:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 07:16:43.682782 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:18:18.570588 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:18.580240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341472e-f657-49ed-a5e4-833ddc2396e4 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:18.583758 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:18.584346 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:18.584624 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:18.584894 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:18.586576 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:18.597074 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:18.598488 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:18.607848 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1312/2607] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:18:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:18:20.006692 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573563b2-6579-4032-9234-9c036f735e16 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:20.016432 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573563b2-6579-4032-9234-9c036f735e16 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:20.019901 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:20.020486 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:20.020755 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:20.021020 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:20.022797 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:20.036063 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:20.038034 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:20.049112 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1296/2608] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:18:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:18:21.230151 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:21.238678 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664a471-5856-42ed-9c3a-ed3556016061 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:21.242737 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:21.243357 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:21.243635 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:21.243911 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:21.245591 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:21.258961 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:21.260465 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:21.271661 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1313/2609] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:18:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:18:27.804873 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:27.806428 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:27.821235 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc00c2-272e-4495-9429-a132ff41c7a3 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:27.827585 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a91af405e24a0b8e9c893e1c299db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:27.828190 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:27.828483 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:27.829037 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:27.831017 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:27.845049 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:27.846676 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:27.858064 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1297/2610] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:18:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:18:31.267203 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=12 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:18:33.274995 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:33.283552 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b3659-4822-4b24-a69c-ebbcb2864036 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:33.287075 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56967a5531ea4e308af5b89c13524c14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:33.287702 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:33.287978 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:33.288264 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:33.289913 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:33.351867 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:33.353235 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:33.362623 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1314/2611] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:18:36.424132 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:36.436850 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab90608-0ac1-4535-aaea-33b1a5985b04 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:36.442668 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:36.443309 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:36.443792 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:36.444399 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:36.446349 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:36.462022 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:36.465695 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:36.490319 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1298/2612] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:18:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:18:39.675648 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:39.677088 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:39.692638 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356d27a-50b9-4838-9cff-0b46a6f39b30 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:39.697345 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:39.698201 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:39.698481 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:39.698749 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] PATH_INFO: `/v3/users/2dda9b6c5edd4dec87e4d23ecfcea5e9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:39.699884 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dda9b6c5edd4dec87e4d23ecfcea5e9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:39.701772 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:39.748993 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dda9b6c5edd4dec87e4d23ecfcea5e9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:39.749564 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dda9b6c5edd4dec87e4d23ecfcea5e9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:39.756550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dda9b6c5edd4dec87e4d23ecfcea5e9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:39.757196 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dda9b6c5edd4dec87e4d23ecfcea5e9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:39.769014 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dda9b6c5edd4dec87e4d23ecfcea5e9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:39.782240 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1315/2613] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:18:39 2022] DELETE /identity/v3/users/2dda9b6c5edd4dec87e4d23ecfcea5e9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:18:39.950946 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:39.964240 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6e711-8b5a-4188-bf02-236a153179f7 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:40.021642 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:40.022186 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:40.022464 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:40.022735 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] PATH_INFO: `/v3/projects/27c9c9c8efd64131b5944c5cf47553b3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:40.023848 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c9c9c8efd64131b5944c5cf47553b3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:40.026291 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:40.071653 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c9c9c8efd64131b5944c5cf47553b3', 'request_id': 'req-b972717b-cb76-4efe-9b2c-e8074be55b27'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.077076 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c9c9c8efd64131b5944c5cf47553b3', 'request_id': 'req-b972717b-cb76-4efe-9b2c-e8074be55b27'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.085144 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c9c9c8efd64131b5944c5cf47553b3', 'request_id': 'req-b972717b-cb76-4efe-9b2c-e8074be55b27'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.096509 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] The token cache is being invalidate because project 27c9c9c8efd64131b5944c5cf47553b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:18:40.096794 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.097448 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.098917 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1299/2614] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:18:39 2022] DELETE /identity/v3/projects/27c9c9c8efd64131b5944c5cf47553b3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:18:40.103473 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:40.104855 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:40.117708 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da567b-73fb-4c81-a7e3-090245781821 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:40.172766 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:40.173389 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:40.173671 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:40.173947 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] PATH_INFO: `/v3/users/20c0a681cf024f55904c2a4ed862b2e6` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:40.175056 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c0a681cf024f55904c2a4ed862b2e6)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:40.176999 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:40.182133 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:18:40.224691 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c0a681cf024f55904c2a4ed862b2e6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.225409 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c0a681cf024f55904c2a4ed862b2e6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.232225 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c0a681cf024f55904c2a4ed862b2e6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.232898 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c0a681cf024f55904c2a4ed862b2e6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.244404 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c0a681cf024f55904c2a4ed862b2e6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.256609 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1316/2615] 10.222.0.36 () {58 vars in 1224 bytes} [Mon Oct 17 07:18:40 2022] DELETE /identity/v3/users/20c0a681cf024f55904c2a4ed862b2e6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 07:18:40.424754 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:18:40.426219 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:18:40.443419 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972717b-cb76-4efe-9b2c-e8074be55b27 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:18:40.491606 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:18:40.503367 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be01c4fc32bc463e9e081f13d9758f92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86c1b532428347929d42c84633169ba2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:18:40.504152 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:18:40.504429 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:18:40.504694 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] PATH_INFO: `/v3/projects/9bc7f1ee12744f5db14f38236b58b790` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:18:40.505772 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc7f1ee12744f5db14f38236b58b790)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:18:40.507717 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:18:40.552292 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc7f1ee12744f5db14f38236b58b790', 'request_id': 'req-b947fb5f-3187-48ca-bcd9-a69c60a3f524'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.557981 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc7f1ee12744f5db14f38236b58b790', 'request_id': 'req-b947fb5f-3187-48ca-bcd9-a69c60a3f524'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.565646 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc7f1ee12744f5db14f38236b58b790', 'request_id': 'req-b947fb5f-3187-48ca-bcd9-a69c60a3f524'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.577317 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] The token cache is being invalidate because project 9bc7f1ee12744f5db14f38236b58b790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 07:18:40.577604 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.578224 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b947fb5f-3187-48ca-bcd9-a69c60a3f524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 07:18:40.580036 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1300/2616] 10.222.0.36 () {58 vars in 1233 bytes} [Mon Oct 17 07:18:40 2022] DELETE /identity/v3/projects/9bc7f1ee12744f5db14f38236b58b790 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 07:18:50.253053 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:18:50.574394 dv-cinder-845069-5-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 07:21:24.721550 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 07:21:24.722937 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:21:24.857727 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc22fc7-c559-4909-beb2-2ec90f73c1e5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 07:21:24.913877 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c01fa-0b43-4d7d-bc8c-365d9c960959 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '522ff98d9ecb4d90bd97a19f71a400c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1f20ff4c5f24755b1f918e07e8b23fc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 07:21:24.914466 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c01fa-0b43-4d7d-bc8c-365d9c960959 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 07:21:24.914739 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c01fa-0b43-4d7d-bc8c-365d9c960959 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 07:21:24.915003 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c01fa-0b43-4d7d-bc8c-365d9c960959 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 07:21:24.916705 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-880c01fa-0b43-4d7d-bc8c-365d9c960959 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 07:21:24.927952 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c01fa-0b43-4d7d-bc8c-365d9c960959 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 07:21:24.929492 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c01fa-0b43-4d7d-bc8c-365d9c960959 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 07:21:24.939040 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1317/2617] 10.222.0.36 () {60 vars in 1365 bytes} [Mon Oct 17 07:21:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 07:21:34.934857 dv-cinder-845069-5-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}